Vision Transformers (ViT)

Layer Scale

Regularization technique introduced in deep ViTs where learnable weights are applied to residual outputs to stabilize the training of initial layers.

← Tillbaka