AI Glossary
The complete dictionary of Artificial Intelligence
CART (Classification and Regression Trees)
Fundamental algorithm for building decision trees using the Gini index for classification and variance for regression.
ID3 (Iterative Dichotomiser 3)
First decision tree algorithm using information gain as node splitting criterion
C4.5 and C5.0
Evolution of ID3 handling continuous attributes and using gain ratio to avoid bias towards attributes with many values.
Bagging (Bootstrap Aggregating)
Ensemble technique creating multiple models on bootstrap subsamples, foundation of Random Forests.
Feature Importance
Method for evaluating the importance of variables based on impurity reduction or permutation in Random Forests.
Out-of-Bag Error
Error estimation without cross-validation using non-selected samples in the bootstrap of each tree.
Tree Pruning (Élagage)
Tree complexity reduction techniques to avoid overfitting (pre-pruning and post-pruning).
Gini Impurity
Measure of the impurity of a node calculating the probability of misclassification of a randomly chosen element.
Information Gain and Entropy
Division criteria based on information theory measuring the entropy reduction after a partition.
Extra-Trees (Extremely Randomized Trees)
Variante des Random Forests avec sélection aléatoire des seuils de division pour réduire encore plus la variance.
Isolation Forest
Application des Random Forests pour la détection d'anomalies en isolant les observations dans des arbres courts.
Gradient Boosted Trees
Méthode d'ensemble séquentielle construisant des arbres pour corriger les erreurs résiduelles des modèles précédents.