Vision Transformers (ViT)

Distillation Token

Additional token in DeiT that learns to mimic the predictions of a teacher model (often a CNN), facilitating knowledge transfer and improving performance with less data.

← Back