Sparse Attention

Linformer

Architecture that projects the key-value matrix into a lower-dimensional space, transforming complexity from O(n²) to O(n*k) where k << n.

← Back