AI-woordenlijst
Het complete woordenboek van kunstmatige intelligentie
Learned Positional Embeddings
Approach where position vectors are parameters learned during training, unlike fixed sinusoidal functions.
Positional Index
Integer numerical value representing the position of a token in a sequence, used as the basis for calculating Positional Encoding.
NoPE (No Positional Encoding)
Experimental approach exploring the performance of Transformers without any form of explicit positional encoding.
2D Positional Encoding
Extension of Positional Encoding for two-dimensional structured data, such as images, by encoding (x, y) coordinates.
Causal Positional Encoding
Positional Encoding used in autoregressive decoder models where attention is masked to be only causal (to the left).
Positional Dropout
Regularization technique applying a dropout layer specifically on position embeddings to avoid over-reliance on them.