Cross-Attention

Relative Positional Attention

Attention mechanism that directly incorporates relative distances between positions in the calculation of attention scores, providing better generalization to variable sequence lengths.

← Indietro