AI Glossary
The complete dictionary of Artificial Intelligence
Contractive Autoencoder (CAE)
A type of autoencoder whose loss function includes a penalty on the norm of the encoder's Jacobian matrix, forcing the latent representation to be insensitive to small input variations.
Jacobian Penalty
Regularization term added to the loss function of a contractive autoencoder, calculated as the sum of squares of the partial derivatives of the latent representation with respect to each input pixel.
Robustness to Perturbations
Ability of a model, particularly a contractive autoencoder, to maintain stable performance in the face of slight modifications or noise in the input data.
Contractive Loss Function
Objective function combining the standard reconstruction error and the Jacobian penalty, optimized during the training of a contractive autoencoder.
Gradient Vanishing
Potential problem when computing the Jacobian penalty in deep networks, where gradients can become extremely small, making optimization difficult.
Contracted Latent Space
The low-dimensional representation space produced by the encoder of a CAE, characterized by low sensitivity to local input variations.
Regularization Factor (Lambda)
Hyperparameter that controls the relative importance of the Jacobian penalty compared to the reconstruction error in the loss function of a contractive autoencoder.
Factor Disentanglement
Objective associated with contractive autoencoders where the latent representation aims to capture the most relevant variation factors of the data while ignoring non-informative variations.
Denoising Autoencoder
Related model that learns to reconstruct a clean input from a corrupted version, sharing the robustness objective with the contractive autoencoder but through a different approach.
Model Sensitivity
Measure of the variation in a model's output (here, the latent representation) in response to small changes in its input, which the contractive autoencoder seeks to minimize.
Regularization by Constraint
Regularization strategy used in CAEs, where an explicit constraint (the penalty on the Jacobian) is imposed on the model parameters to guide its learning.