AI Glossary
The complete dictionary of Artificial Intelligence
Noisy Bayesian Optimization
Extension of Bayesian optimization designed to handle observations of the objective function contaminated by random noise, requiring robust surrogate models and acquisition strategies.
Heteroscedastic Surrogate Model
Gaussian process or other regression model that explicitly models the noise variance as a function of input variables, allowing finer uncertainty management in Bayesian optimization.
Robust Acquisition Strategy
Modified acquisition function to be less sensitive to noisy observations, such as Expected Improvement with noise integration or Knowledge Gradient, aiming to stabilize the optimization process.
Replication Averaging
Technique involving evaluating the same candidate point multiple times and using the average of results to reduce the impact of noise, at the cost of increased evaluation budget.
Noisy Thompson Sampling
Variant of the Thompson Sampling acquisition strategy adapted to the noisy context, where a sample is drawn from the posterior predictive distribution that includes noise to guide the next evaluation.
Noise Calibration
Process of estimating the parameters of the noise model (e.g., variance in a homoscedastic case or variance function in a heteroscedastic case) from observed data.
GP-UCB with Noise
Adaptation of the Gaussian Process Upper Confidence Bound algorithm that incorporates noise variance into the upper confidence bound calculation, ensuring theoretical regret guarantees in the presence of noise.
Particle Filter for BO
Alternative approach to Gaussian processes using particle filters to model the posterior distribution of the objective function, offering greater flexibility for modeling non-Gaussian noise.
Noisy Multi-Fidelity Optimization
Framework where low-fidelity evaluations, often more noisy, are combined with high-fidelity evaluations to accelerate optimization while managing different noise levels.
Cumulative Regret in the Presence of Noise
Performance metric evaluating the sum of differences between the value of the best solution found and the optimal (expected) value of the objective function, accounting for the impact of noise on decisions.
Markov Chain Monte Carlo (MCMC) for Inference
Bayesian inference technique, such as Gibbs sampling or Hamiltonian Monte Carlo, used to estimate the posterior distribution of surrogate model hyperparameters in the presence of noise.
Heteroscedastic Noise Kriging
Advanced form of kriging (Gaussian process model) that allows the noise level to vary spatially, providing more realistic modeling for many real-world applications.
Quantile-Based Acquisition Strategy
Family of acquisition functions that target specific quantiles (e.g., the lower quantile) of the predictive distribution, to guard against optimistic evaluations due to noise.
AR(1) Type Noise Model
Modeling noise as an autoregressive process of order 1, capturing temporal correlation in evaluation errors, relevant for sequential optimizations where evaluations are not independent.
Noisy Bayesian Optimization with Unknown Constraints (BOUC)
Extension of Bayesian optimization where constraints are unknown and must be learned from noisy evaluations, adding a layer of complexity to uncertainty management.