Attention Scaling

Query-Key Dimensionality

Common dimension of query and key vectors in multi-head attention, whose square root determines the normalization scaling factor.

← 뒤로