AI Glossary
The complete dictionary of Artificial Intelligence
Successive Halving
Progressive elimination method that halves the number of configurations at each iteration by allocating more resources to the identified best performances.
Dynamic resource allocation
Adaptive strategy that distributes computational budget based on observed performances of hyperparameter configurations during optimization.
Computational budget
Total available computational resource defined in terms of iterations, training epochs, or processor time allocated to hyperparameter optimization.
Early elimination
Mechanism that prematurely stops the evaluation of underperforming configurations to save computational resources.
ASHA
Asynchronous Successive Halving Algorithm implements Successive Halving asynchronously to maximize resource utilization on parallel architectures.
Multi-fidelity optimization
Approach using lower-quality but faster approximations to quickly evaluate configurations before their full evaluation.
Hyperparameter configuration
Specific set of values assigned to the hyperparameters of a machine learning model defining a unique configuration to evaluate.
Bracket elimination
Staggered elimination process in Hyperband where multiple brackets with different initial resource allocations run in parallel.
Resource allocation strategy
Policy determining the distribution of computational budget among hyperparameter configurations during the optimization process.
Hyperparameter space exploration
Systematic process of searching in the hyperparameter space to identify optimal configurations according to defined performance metrics.
Performance-based pruning
Pruning technique for configurations based on their relative performance compared to a threshold or the best observed configurations.
Computational efficiency
Measure of how efficiently the optimization algorithm uses available resources to find performant configurations.
Configuration fidelity
Level of precision or quality used to evaluate a hyperparameter configuration, progressively increased for the best configurations.
Adaptive resource allocation
Mechanism that dynamically adjusts resource distribution based on observed performance trends during optimization.