Vision Transformers (ViT)

DeiT (Data-efficient Image Transformer)

Variant of ViT trainable with more modest amounts of data through a knowledge distillation strategy where a distillation token is added to learn from a CNN teacher.

← Tillbaka