AI Glossary
The complete dictionary of Artificial Intelligence
Causal Language Models
Predictive Transformer architecture generating text token by token by conditioning each prediction on previous tokens, used in models like GPT.
Masked Language Models
Self-supervised approach where tokens are masked in the input and the model must predict them, enabling bidirectional context learning.
Vision-Transformer (ViT)
Adaptation of the Transformer architecture for image processing, replacing convolutions with attention mechanisms on image patches.
Multimodal Models
Transformers capable of processing and generating multiple types of data (text, image, audio) simultaneously using shared embedding spaces.
Token Embeddings
Dense vector representations of input tokens, learned during pre-training and capturing semantic and syntactic information.
Attention Scale
Parameter controlling the distribution of attention in Transformer models, influencing the concentration or dispersion of contextual focus.
Transformer-based Diffusion Models
Integration of Transformer architecture into the iterative denoising process of diffusion models, improving overall generation coherence.
Attention Backpropagation
Mechanism allowing analysis and interpretation of Transformer model decisions by visualizing attention weights between tokens.
Foundation Models
Large Transformer models pre-trained on massive data, serving as a starting point for various applications through fine-tuning or prompting.