Positional Encoding

Alibi Positional Encoding

Method that penalizes attention scores based on the distance between tokens, enabling effective extrapolation to longer sequence lengths without retraining.

← Tillbaka