Tree Ensembles

Decision trees and ensemble methods (random forests and gradient boosting). The dominant approach for structured/tabular data.

Notes

• Tree Ensembles — decision trees, random forests, gradient boosting, XGBoost, LightGBM, hyperparameters
• Tree Ensembles — Implementation — XGBoost, LightGBM, RandomForest with scikit-learn, early stopping, SHAP feature importance

Links

← Linear Models and GLMs → Probabilistic Models

• SHAP and Feature Attribution
• Gradient Descent