Attention Mechanism

ALiBi Position Embedding

Positional encoding method that applies linearly decreasing biases based on distance, enabling better extrapolation to longer sequence lengths.

← Wstecz