Attention Head Analysis

Positional Role

Function of an attention head that primarily focuses on relative positional relationships between tokens, helping the model understand word order regardless of their semantic content.

← Indietro