KI-Glossar
Das vollständige Wörterbuch der Künstlichen Intelligenz
Attention Mechanism
Fundamental component enabling transformers to weigh the importance of different parts of a sequence
Encoders and Decoders
Basic architectural structure of transformers with two main components for processing and generating sequences
Positional Encoding
Technique enabling the incorporation of positional information into embeddings without using recurrence
Multi-Head Attention
Extension of the attention mechanism using multiple attention heads in parallel to capture different types of relationships
BERT
Revolutionary bidirectional pre-trained model based on the transformer encoder
GPT
Series of generative models based on the transformer decoder for text generation
Vision Transformers
Applying the transformer architecture to computer vision tasks by treating images as sequences
Self-Attention
Mechanism allowing each element of a sequence to interact with all other elements of the same sequence
Cross-Attention
Attention mechanism between two different sequences, essential in translation tasks
Transformer-XL
Extension of transformers capable of modeling long-term dependencies without context fragmentation
T5
Text-to-text model unifying all NLP tasks within the same text input-output format
Sparse Attention
Efficient attention variants reducing computational complexity by limiting connections
Layer Normalization
Essential normalization technique for stabilizing the training of deep transformers
Feed-Forward Networks
Fully connected networks applied at each position in transformer layers
Attention Masks
Mechanism allowing control over which tokens can attend to one another