AI Glossary
The complete dictionary of Artificial Intelligence
Bagging Bootstrap Aggregating
Ensemble technique creating multiple models on bootstrap samples and combining their predictions by majority vote or average.
Random Forest
Bagging algorithm using decision trees with random feature selection at each split to reduce correlation between models.
Extra Trees Extremely Randomized Trees
Variant of Random Forest adding extra randomization in the selection of split thresholds to further reduce variance.
Pasting Ensemble
Ensemble method similar to bagging but using subsets without replacement of the training data.
Voting Classifiers
Technique combining several heterogeneous classifiers using hard majority vote or weighted soft average for the final prediction.
Stacking Stacked Generalization
Ensemble method training a meta-model to combine predictions from multiple base models using cross-validation.
Blending
Simplified variant of stacking using a hold-out validation set to train the meta-model instead of cross-validation.
Out-of-Bag Error Estimation
Internal evaluation method for bagging techniques using the non-selected samples (out-of-bag) to estimate the generalization error.
Feature Importance in Ensembles
Techniques for evaluating variable importance in ensemble models based on impurity reduction or permutation.
Bootstrap Sampling Methods
Advanced bootstrap sampling techniques including balanced bootstrap, stratified bootstrap, and weighted bootstrap for datasets.
Isolation Forest
Anomaly detection algorithm based on Random Forest using the average path length in trees to measure point isolation.
Rotation Forest
Extension of Random Forest applying PCA transformations on feature subsets before training each tree.
Bootstrap Aggregating Regressors
Application of bagging to regression problems combining predictions by mean or median to reduce variance.
Balanced Random Forest
A variant of Random Forest that handles imbalanced classes through balanced bootstrap sampling for each tree.
Quantile Regression Forest
Extension of Random Forest to estimate conditional quantiles of the target variable distribution in regression.