Residual normalization layer applied after the attention mechanism, combining the attention output with the original input (residual connection) before normalizing the sum.

📖

termini

Linear Projection in Attention

Linear transformation (multiplication by a weight matrix) applied to input embeddings to generate Query, Key, and Value vectors, allowing the model to learn attention-specific representation spaces.

🔍

Glossario IA

Add & Norm Layer

Linear Projection in Attention

Nessun risultato trovato