Attention Masks

Key Padding Mask

Specific mask applied to keys in the attention mechanism to prevent padding tokens from influencing attention scores, typically added before the softmax operation.

← Indietro