AI Glossary
The complete dictionary of Artificial Intelligence
Quantization-Aware Training (QAT)
Deep learning model training method simulating quantization during the learning process to optimize post-quantization performance.
Fake Quantization
Operation simulating the effects of quantization during training by rounding values while maintaining gradients for backpropagation.
Quantization Range
Value interval [min, max] used to map floating-point numbers to quantized integers, determining the precision of the representation.
Symmetric Quantization
Quantization technique where the interval is centered around zero, simplifying calculations but potentially reducing efficiency for asymmetric distributions.
Asymmetric Quantization
Quantization method using a zero point different from zero, optimizing the use of dynamic range for non-centered distributions.
Dynamic Range Quantization
Technique dynamically adapting quantization ranges during execution to optimize the use of available bits.
Per-Tensor Quantization
Method applying a single set of quantization parameters to an entire tensor, simplifying implementation.
Integer-Only Quantization
Approach completely eliminating floating-point operations, requiring specialized techniques to maintain model precision.
Layer-wise Quantization
Strategy optimizing the quantization of each layer individually according to its specific characteristics and sensitivity.
Quantization Sensitivity Analysis
Evaluation of the impact of quantization on each component of the model to identify layers requiring particular attention.
Quantization-Aware Training Loop
Modified training cycle integrating quantization simulation operations at each forward and backward pass.
Batch Folding
Optimization technique merging batch normalization parameters with convolutional weights before quantization.
Gradient Clipping in QAT
Method limiting the amplitude of gradients during quantized training to stabilize convergence despite approximations.
Stepped Quantization
Progressive approach gradually increasing the level of quantization during training to facilitate model adaptation.