AI Glossary
The complete dictionary of Artificial Intelligence
Activation Quantization
Process of reducing the precision of activation values propagated through the neural network, essential for minimizing memory usage and optimizing computations on resource-constrained microcontrollers.
Quantization-Aware Training
Approach where quantization is simulated during the training phase to minimize accuracy loss, resulting in more robust models once quantized for embedded devices.
8-bit Precision
Numerical representation format using 8 bits per parameter, offering an optimal balance between precision and efficiency for most deep learning applications on IoT devices.
Neural Network Pruning
Compression technique that selectively removes the least important weights or neurons from the network, significantly reducing model size while preserving essential performance.
Extreme Binarization
Extreme form of quantization that reduces all weights and activations to 1 bit (+1/-1), maximizing compression and drastically accelerating computations on specialized IoT hardware.
Fixed-Point Representation
Numerical format where numbers are represented with a fixed number of bits for the integer and fractional parts, preferred in IoT devices for its hardware simplicity and energy efficiency.
Edge AI Optimization
Set of techniques combining quantization, compression, and algorithmic optimization to efficiently adapt AI models to the strict constraints of edge and IoT devices.
Structured Weight Pruning
Pruning variant that removes entire structures (filters, channels, or attention heads) rather than individual weights, generating more efficient models on IoT hardware.
Sub-8-bit Quantization
Advanced techniques reducing precision below 8 bits (4, 2, or even 1 bit) for maximum compression, suitable for extremely constrained IoT applications.
Tensor Factorization
Mathematical technique decomposing large-dimensional weight tensors into products of smaller tensors, drastically reducing the number of parameters for IoT deployment.
Compressed Weight Encoding
Compression algorithm applied after quantization using techniques like Huffman or range encoding to further reduce model storage size on IoT devices.