Glossario IA
Il dizionario completo dell'Intelligenza Artificiale
Teacher-Student Model
Two-tier architecture where a large expert model guides the training of a compact model, preserving performance while reducing computational resources.
Logit Distillation
Knowledge transfer method based on direct alignment of logits (pre-activations) between teacher and student, preserving decision margins and numerical relationships.
Relation Distillation
Technique preserving structural relationships between samples or features, including similarity graphs, pairwise distances, and inter-feature correlations.
Knowledge Distillation Loss
Loss function combining KL-divergence distillation between soft distributions and standard cross-entropy loss with hard labels, weighted by an alpha parameter.
FitNets
Distillation architecture introducing intermediate layers (hint layers) to guide student learning by aligning representations at different depths.
Knowledge Consolidation
Process preserving acquired knowledge during continual learning, using distillation to transfer historical expertise to updated models.