Glossario IA
Il dizionario completo dell'Intelligenza Artificiale
Teacher model
Large-scale pre-trained neural network serving as a knowledge source, whose probabilistic outputs are used to guide the student model's learning.
Student model
Compact model architecture optimized to reproduce the teacher model's behavior, aiming to maintain performance while reducing computational complexity.
Proxy task
Artificially designed pretext task for generating supervised learning signals from unlabeled data, serving as an intermediate objective for self-supervision.
Distillation loss
Objective function measuring the divergence between the teacher and student probability distributions, typically implemented using Kullback-Leibler divergence.
Softmax temperature
Scaling parameter applied to the softmax function to soften probability distributions, thereby revealing the relationships between classes that the teacher model has learned.
Distribution alignment
Technique aiming to minimize the gap between probability distributions predicted by the teacher and student in feature or prediction space.
Pretext task
Secondary self-supervised task designed to force the model to learn useful representations, such as predicting masked parts or reconstructing corrupted inputs.