Sparse Attention

Performer

Model based on FAVOR+ attention that efficiently approximates softmax attention through positive orthogonal random features, enabling linear complexity.

← Tillbaka