Słownik AI

📖

pojęcia

Learned Positional Embeddings

Approach where position vectors are parameters learned during training, unlike fixed sinusoidal functions.

📖

pojęcia

Positional Index

Integer numerical value representing the position of a token in a sequence, used as the basis for calculating Positional Encoding.

📖

pojęcia

NoPE (No Positional Encoding)

Experimental approach exploring the performance of Transformers without any form of explicit positional encoding.

📖

pojęcia

2D Positional Encoding

Extension of Positional Encoding for two-dimensional structured data, such as images, by encoding (x, y) coordinates.

📖

pojęcia

Causal Positional Encoding

Positional Encoding used in autoregressive decoder models where attention is masked to be only causal (to the left).

📖

pojęcia

Positional Dropout

Regularization technique applying a dropout layer specifically on position embeddings to avoid over-reliance on them.