KI-Glossar
Das vollständige Wörterbuch der Künstlichen Intelligenz
KV Cache
Inference optimization that caches the keys and values of previous tokens to avoid recalculating attention states at each new token generation.
RLHF (Reinforcement Learning from Human Feedback)
Alignment paradigm where a model is fine-tuned through reinforcement learning using rewards derived from human preferences to calibrate its behavior.
Multi-head Attention Mechanism
Extension of attention where multiple heads compute attention representations in parallel on different projected subspaces, allowing the model to focus on various aspects of the sequence.
Decoder-Only
Transformer architecture consisting exclusively of decoder blocks with causal masking, optimized for autoregressive language modeling and generation tasks.
Probability Density Modeling
Fundamental objective of language models that learn to estimate the conditional probability P(token_t | tokens_<t) for each position in a sequence.