When you train a Random Forest you don’t get a Random Forest. You get Bagged Trees — at least when using RandomForestRegressor in scikit-learn. Bagged trees are like Random Forests but without the sampling mechanism for the features. The default value of the hyperparameter
Great post, got me thinking and taking various notes. Thank you for sharing!
Tow feedback points:
The current default for max_features is sqrt, but yes, it used to be auto. https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html
I think the post could benefit from a more obvious explaination of what that max_features hyperparameter actually does.
Thanks for the feedback.
max_features is sqrt per default for RandomForestClassifier, but it's 1.0 (all features) for RandomForestRegressor: https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestRegressor.html