Attention Scaling

Embedding Dimension Normalization

Normalization technique based on embedding dimensionality to ensure comparable magnitude of vector representations in the attention space.

← Indietro