AI Glossary
The complete dictionary of Artificial Intelligence
Adapter Layers
Lightweight neural modules inserted between the layers of a pre-trained model to adapt it to new tasks. These additional layers are the only ones trained during fine-tuning, thus preserving the original knowledge of the model.
Progressive Fine-tuning
Adaptation strategy where fine-tuning starts with the upper layers of the model and then progressively moves to the lower layers. This approach helps preserve fundamental representations while adapting specialized layers.
Task-specific Head
Additional module added to a pre-trained model to adapt it to a specific task such as classification or generation. This head is typically trained separately or in combination with partial fine-tuning of the model.
Layer Freezing
Technique involving keeping the weights of certain layers of a model unchanged during fine-tuning. This approach allows preserving fundamental knowledge while adapting only the upper layers to the new task.