AI Glossary
The complete dictionary of Artificial Intelligence
Hyperparameter Optimization
Automated process of searching for the best hyperparameters for a learning model using techniques like Bayesian optimization, random search, or meta-learning.
Configuration Space
Structured set of all possible hyperparameter combinations for a learning algorithm, including constraints, dependencies, and valid value ranges.
Surrogate Model
Approximate model used in Bayesian optimization to estimate the expensive-to-evaluate performance function, allowing for efficient exploration of the hyperparameter space.
Warm-Starting
Initialization technique for hyperparameter optimization using knowledge from similar tasks or previous optimizations to accelerate convergence.
Base-Learner
Machine learning model whose hyperparameters are optimized by the meta-learning system, serving as the target for configuration recommendations.
Performance-Based Meta-Learning
Approach that uses the historical performance of configurations on different tasks to learn how to predict the best configurations for new, similar tasks.
Multi-Fidelity Optimization
Optimization strategy that uses low-cost (low-fidelity) approximations to quickly evaluate configurations before validating the most promising ones with high-fidelity evaluations.
Meta-Dataset
Structured collection of metadata about multiple learning tasks, including dataset features and the performance of hyperparameter configurations.
Few-Shot Hyperparameter Optimization
Meta-learning approach that allows optimizing hyperparameters with very few evaluations on the target task by transferring knowledge from a large number of source tasks.
Acquisition Function
Function used in meta-learning guided Bayesian optimization to balance exploration and exploitation by selecting hyperparameter configurations to evaluate.