Positional Encoding

Causal Positional Encoding

Positional Encoding used in autoregressive decoder models where attention is masked to be only causal (to the left).

← Geri