AI Glossary
The complete dictionary of Artificial Intelligence
Constrained Bayesian Optimization
Optimization method that combines Bayesian inference with explicit constraints on hyperparameters, using Gaussian processes to model the objective function while respecting the imposed bounds and restrictions.
Hierarchical Grid Search
Structured optimization approach that exploits dependencies between hyperparameters by organizing the search according to a hierarchical tree, thus eliminating invalid combinations and optimizing the search space.
Pareto Multi-objective Optimization
Advanced technique that identifies the set of Pareto-optimal solutions when multiple conflicting objectives must be optimized simultaneously under hyperparameter constraints, enabling a balanced trade-off between performance and resources.
Constrained Genetic Algorithms
Evolutionary metaheuristics adapted to manage optimization constraints through penalty mechanisms, repair, or constrained dominance, preserving genetic diversity while respecting search space restrictions.
Bounded Particle Swarm Optimization
Variant of the PSOP algorithm where particles evolve in a search space delimited by explicit constraints, using reflection or reset mechanisms to maintain solutions in admissible regions.
Projected Gradient Methods
Optimization techniques that project the gradient onto the tangent space of active constraints, ensuring that each iteration remains in the feasible region while following the steepest descent direction.
Constrained Simulated Annealing
Metaheuristic algorithm that integrates constraint management mechanisms in the process of accepting new solutions, modifying the objective function through adaptive penalties or using repair operators.
Constrained SMBO
Sequential Model-Based Optimization extended to handle hyperparameter constraints, building probabilistic models for the objective and constraints simultaneously to effectively guide the search towards feasible solutions.
Constrained Differential Evolution Optimization
Continuous optimization algorithm adapted for constrained problems, using specific mutation and crossover operators combined with selection methods based on feasibility and constrained dominance.
Interior Point Methods
Optimization algorithms that transform constraints into barrier penalties, exploring the interior of the feasible region and converging to the optimal solution while strictly maintaining all active constraints.
Augmented Lagrangian Optimization
Method that combines Lagrangian duality with quadratic penalty terms, transforming a constrained problem into a sequence of unconstrained problems while guaranteeing convergence to feasible optimal solutions.
TPE with Dependencies
Modified Tree-structured Parzen Estimator to handle conditional dependencies between hyperparameters, building hierarchical probabilistic models that respect the conditional structure of the search space.
Robust Hyperparameter Optimization
Approach that seeks high-performing hyperparameter configurations under various conditions and uncertainties, incorporating robustness constraints to ensure model stability against variations in input data.
Constrained Multi-fidelity Optimization
Strategy that uses evaluations at different fidelities under budget constraints, intelligently allocating resources between fast approximations and accurate evaluations to accelerate convergence to optimal hyperparameters.
Hyperparameter Optimization with Resource Constraints
Optimization framework that explicitly integrates computational, memory, and time limitations into the search process, using early stopping and adaptive resource allocation mechanisms.
Constrained Expected Improvement
Acquisition criterion for Bayesian optimization that modifies the standard expected improvement to penalize solutions violating constraints, balancing exploration of new regions with adherence to imposed restrictions.
Optimization with Categorical Constraints
Specialized methods for handling categorical hyperparameters with mutual restrictions or exclusions, using adapted encodings and search operators that respect the discrete structure of the space.
Neural Architecture Search with Constraints
Process of automating neural architecture design that incorporates constraints on complexity, latency, or energy consumption, ensuring that generated architectures respect hardware and operational specifications.