AI Glossary
The complete dictionary of Artificial Intelligence
Pre-pruning
Pruning technique that stops the growth of the decision tree before it reaches its maximum size by applying predefined stopping criteria.
Post-pruning
Pruning method that consists of first building a complete tree and then reducing its complexity by eliminating non-essential branches.
Cost complexity pruning
Pruning technique that minimizes a cost function combining classification error and tree complexity through an alpha parameter.
Reduced error pruning
Pruning method that removes nodes if it does not increase the classification error on a separate validation set.
Minimum description length
Pruning principle based on information theory that favors models offering the best compromise between simplicity and predictive power.
Pessimistic error pruning
Technique that estimates future error by adding a statistical penalty to the observed error to avoid overfitting.
Error-based pruning
Family of pruning algorithms that use different error measures to decide which branches to remove.
Minimum error pruning
Algorithm that recursively removes nodes whose removal minimizes the expected error on test data.
Bottom-up pruning
Pruning approach that starts from the tree leaves and progresses towards the root by evaluating each node for potential removal.
Top-down pruning
Pruning method that evaluates nodes from the root to the leaves, removing entire subtrees when deemed necessary.
Alpha parameter
Regularization parameter in cost complexity pruning that controls the trade-off between tree size and classification error.
Pruning path
Sequence of decreasing complexity trees generated during the pruning process, each tree being a subtree of the previous one.
Weakest link pruning
Variant of cost complexity pruning that identifies and iteratively eliminates branches with the weakest impact on overall performance.
Cross-validation pruning
Technique that uses cross-validation to determine the optimal pruning level and avoid overfitting.
Critical value pruning
Method that removes branches whose test statistic falls below a predetermined critical threshold.
Cost-sensitive pruning
Pruning approach that takes into account the different costs associated with classification errors to optimize the tree structure.
Optimal pruning
Process that guarantees finding the optimal subtree according to a given criterion, often implemented by algorithms like CART.