Positional Encoding

T5's Relative Position Bias

Specific implementation of relative position bias used in the T5 model, where biases are learned for different relative distances between tokens in attention scores.

← Back