AI Glossary
The complete dictionary of Artificial Intelligence
162
categories
2,032
subcategories
23,060
terms
terms
Learned Positional Embeddings
Approach where position vectors are parameters learned during training, unlike fixed sinusoidal functions.
terms
Positional Index
Integer numerical value representing the position of a token in a sequence, used as the basis for calculating Positional Encoding.
terms
NoPE (No Positional Encoding)
Experimental approach exploring the performance of Transformers without any form of explicit positional encoding.
terms
2D Positional Encoding
Extension of Positional Encoding for two-dimensional structured data, such as images, by encoding (x, y) coordinates.
terms
Causal Positional Encoding
Positional Encoding used in autoregressive decoder models where attention is masked to be only causal (to the left).
terms
Positional Dropout
Regularization technique applying a dropout layer specifically on position embeddings to avoid over-reliance on them.
🔍