AI Glossary
The complete dictionary of Artificial Intelligence
Bagging
Ensemble technique creating multiple models on bootstrap samples to reduce variance
Boosting
Sequential method where each model corrects the errors of the previous one to reduce bias
Stacking
Combination of models via a meta-model that learns to weight their predictions
Random Forest
Ensemble of decision trees using bagging and random feature selection
Gradient Boosting
Boosting that optimizes a loss function by gradient descent in function space
AdaBoost
Adaptive boosting algorithm that weights examples according to previous errors
XGBoost
Optimized implementation of gradient boosting with regularization and parallelization
LightGBM
Gradient boosting using growing leaf-wise and feature binning for speed
CatBoost
Gradient boosting specialized in the automatic processing of categorical variables
Voting Classifiers
Aggregation of predictions by hard majority voting or soft averaging of probabilities
Blending
Technique similar to stacking using hold-out validation to train the meta-model
Dynamic Ensemble Selection
Dynamic selection of model subsets based on the characteristics of each instance
Ensemble Pruning
Reducing ensemble size by selecting the most relevant models
Bayesian Model Averaging
Bayesian approach combining models according to their posterior probabilities
Mixture of Experts
Architecture où des réseaux experts sont activés par un réseau de gating