Layer Normalization

Position Encoding

Technique that adds absolute or relative position information to input embeddings to allow the model to understand the order of tokens in a sequence.

← Geri