AI 용어집
인공지능 완전 사전
Auto-regression
Generation process where each token is predicted sequentially based on all previous tokens, enabling progressive and coherent text construction.
Decoder-Only Architecture
Transformer model structure that eliminates encoders to focus solely on the decoder, optimized for text generation using masked attention to prevent future information leakage.
Multi-Head Attention Mechanism
Technique allowing the model to simultaneously focus on different positions in the input sequence through multiple independent attention heads, capturing various types of dependencies.
BPE Tokenization
Byte-Pair Encoding algorithm that segments text into optimal subwords, balancing vocabulary size and semantic coverage for efficient natural language processing.
Causal Attention Mask
Binary matrix applied during attention to prevent each position from attending to future positions, thus preserving the causal nature of text generation.
Model Parameters
Trainable weights of the neural network, whose number characterizes the model's capacity, ranging from millions to billions depending on the desired complexity and performance.
Temperature Sampling
Parameter controlling the degree of randomness in generation, where high values increase diversity and low values favor safer and more coherent predictions.
Context Window
Maximum number of tokens the model can consider simultaneously during generation, determining its ability to maintain coherence over long texts.