KI-Glossar
Das vollständige Wörterbuch der Künstlichen Intelligenz
BERT's Segment Embeddings
Additional embeddings in BERT distinguishing different segments or sentences in a sequence, complementing token and position embeddings.
Position Interpolation
Technique enabling pre-trained models with positional encodings to handle longer sequences through linear interpolation of existing positions.
Length Extrapolation
Ability of a model to generalize to sequence lengths greater than those seen during training, strongly dependent on the positional encoding scheme used.
Learned Positional Bias
Directly learned bias parameters added to attention scores based on relative positions, efficient alternative to full positional embeddings.
Position-Aware Attention
Variant of the attention mechanism explicitly integrating positional information into similarity computations, improving the model's sequential awareness.