Glossario IA

📖

termini

ALiBi (Attention with Linear Biases)

Alternative to positional encoding using linear biases directly in attention scores to represent distances between tokens.

📖

termini

Disentangled Attention

Attention mechanism that explicitly separates content and position contributions in attention score computation.

📖

termini

Multi-scale Positional Encoding

Approach combining multiple scales of positional representation to capture both local and global sequence structures.

📖

termini

Position Dropout

Regularization technique applied specifically to position embeddings to prevent model over-reliance on positional information.