AI Glossary
The complete dictionary of Artificial Intelligence
BOHB (Bayesian Optimization HyperBand)
Hybrid algorithm combining Bayesian optimization for probabilistic modeling and HyperBand for adaptive resource allocation, enabling efficient hyperparameter search.
Surrogate Model
Probabilistic model (typically GP or TPE) that approximates the expensive-to-evaluate objective function to guide hyperparameter search.
Acquisition Function
Function that balances exploration and exploitation to determine the next hyperparameter configurations to evaluate.
TPE (Tree-structured Parzen Estimator)
Non-parametric surrogate model using kernel density estimators to model hyperparameter distributions.
Computational Budget
Limited resource (time, episodes, data) allocated to the evaluation of hyperparameter configurations in BOHB.
Bandit-Based Allocation
Resource allocation strategy inspired by multi-armed bandits to optimize the exploration-exploitation trade-off.
Hyperparameter Importance
Measure of each hyperparameter's impact on model performance, used to prioritize search in BOHB.
Parallel Evaluation
Ability to evaluate multiple hyperparameter configurations simultaneously to accelerate optimization in BOHB.
Thompson Sampling
Method of selection that samples configurations according to their probability of being optimal according to the surrogate model.
Bracket Configuration
Specific parameter setting of HyperBand defining the number of configurations and the evolution of the budget for a given bracket.