Sparse Attention

Low-rank Approximation

Technique approximating the attention matrix through low-rank decomposition, significantly reducing memory and computational requirements.

← Back