Attention Masks

Binary Mask

Matrix containing only 0 and 1 values where 1 indicates positions to keep and 0 those to mask, generally applied through element-wise multiplication before or after the attention softmax.

← Tillbaka