AI Glossary
The complete dictionary of Artificial Intelligence
Weight Sharing
Method where multiple neural connections share the same parameters to significantly reduce the number of unique weights.
Low-Rank Factorization
Decomposition of weight matrices into products of lower-rank matrices to compress dense network layers.
Tensor Decomposition
Advanced technique factorizing convolutional weight tensors into simpler tensors to reduce computational complexity.
Sparse Coding
Representation of activations with many zero coefficients, enabling efficient compression and accelerated computation.
Huffman Coding
Lossless compression algorithm assigning variable-length binary codes to weights based on their frequency of occurrence.
Model Splitting
Division of a model into segments distributed between clients and server to minimize communication while preserving confidentiality.
Parameter Binarization
Conversion of weights into binary values (+1/-1) to drastically reduce memory and accelerate calculations on limited devices.
Federated Averaging
Aggregation algorithm weighting local model updates according to client dataset sizes for global convergence.
Model Pruning Ratio
Percentage of weights or neurons removed from the original model, determining the level of compression applied.
Quantization-aware Training
Training that incorporates the effects of quantization to minimize performance degradation after compression.