AI 용어집
인공지능 완전 사전
Self-Supervised Pre-training
Initial training phase where a model learns general representations on large amounts of unlabeled data before fine-tuning on specific tasks.
Generative Pre-training
Self-supervised approach where the model learns by generating or reconstructing data, developing a deep understanding of the underlying distribution.
Self-Supervised Data Augmentation
Automatic generation of sample variations to create positive/negative pairs used as implicit supervised training signal.
Bidirectional Encoder
Component using past and future context to encode tokens, crucial for masking tasks in generative self-supervision.
Next Sentence Prediction
Pretext task where the model predicts whether two sentences logically follow each other, learning inter-sentence relationships without external supervision.
Causal Language Modeling
Self-supervised task where the model predicts future tokens based solely on past context, unlike bidirectional MLM.
Pretext Task Design
Engineering of optimal pretext tasks to capture relevant data structures without external supervision, key to self-supervised learning success.