Sparse Attention

Reformer

Architecture using locality-sensitive hashing (LSH) to limit attention computations to the most similar pairs, with quasi-linear complexity in sequence length.

← Back