AI Glossary
The complete dictionary of Artificial Intelligence
Stochastic Differential Equation (SDE)
Differential equation that includes a random noise term, used in score-based models to define a continuous diffusion process that transforms data into noise.
Reverse-time SDE
Stochastic process that reverses diffusion using the learned score, allowing generation of samples from noise by going backwards in time.
Noise Conditional Score Network (NCSN)
Neural network architecture that takes both noisy data and noise level as input to predict the score, thus conditioning its prediction on the perturbation scale.
Fisher Divergence
Metric used in score matching to measure the difference between the true score and the score predicted by the model, avoiding calculation of the normalization constant.
Manifold Hypothesis
Principle stating that high-dimensional data lies on a lower-dimensional manifold, a key hypothesis justifying the effectiveness of score-based models.
Score-based Model
Generative model that parameterizes and learns the score vector field of a data distribution, enabling generation of new samples via diffusion processes.
Implicit Score Matching
Training objective for score matching that does not require evaluation of the target data distribution score, making it applicable when the density is unknown.
Exponential Moving Average (EMA) of Scores
Regularization technique where an exponential moving average of the score network weights is maintained during training to stabilize the generation process.
Guided Sampling
Method that modifies the score-based model's sampling process to control the attributes of generated samples, often using a pre-trained classifier.