AI Glossary
The complete dictionary of Artificial Intelligence
Learning Rate (Taux d'apprentissage)
Hyperparameter controlling the contribution of each estimator to the final model, allowing a trade-off between convergence speed and model accuracy.
Max Depth (Profondeur maximale)
Hyperparameter defining the maximum depth of each decision tree, controlling model complexity and the risk of overfitting.
Split Finding
Algorithmic process for finding the best split point in a tree node, optimized in XGBoost through a data structure called histogram.
Tree Pruning (Élagage d'arbre)
Post-pruning technique based on gain score, which removes tree branches that do not provide positive loss gain to simplify the model.
Early Stopping (Arrêt précoce)
Regularization technique that stops training when performance on a validation set stops improving, preventing overfitting.
Gamma (min_split_loss)
Regularization hyperparameter specifying the minimum loss required to make a new split in a tree node, controlling complexity.
Lambda (L2 regularization on weights)
L2 regularization hyperparameter applied to tree leaf weights, reducing their magnitude to prevent overfitting.
Alpha (L1 regularization on weights)
L1 regularization hyperparameter applied to tree leaf weights, promoting sparsity and potentially setting some weights to zero.
Scale-Pos-Weight
Hyperparameter used for imbalanced classification problems, weighting the positive class relative to the negative class.
Parallelization
Ability of XGBoost to parallelize tree construction across multiple CPU cores, significantly speeding up training time.
Cache-Aware Access
Algorithmic optimization in XGBoost that organizes memory access to maximize processor cache utilization, improving performance.