Sparse Attention Mechanisms

LSH Attention

Attention mechanism using locality-sensitive hashing to group similar queries and keys, computing attention only within these groups.

← Terug