AI Glossary
The complete dictionary of Artificial Intelligence
Contrastive Learning
Technique learning representations by maximizing similarity between augmented positive samples and minimizing that of negatives.
Masked Language Modeling
Prediction of masked tokens in text sequences, popularized by BERT and RoBERTa models.
Masked Image Modeling
Prediction of masked image patches using surrounding visual context, adapted from NLP techniques.
Self-Supervised Clustering
Automatic creation of clusters to generate pseudo-labels subsequently used for supervised training.
Rotation Prediction
Proxy task consisting of predicting the rotation angle applied to an image to learn visual features.
Puzzle Reconstruction
Learning by reconstructing images from scrambled patches, developing spatial understanding.
Image Colorization
Automatic prediction of colors for black and white images as a self-supervised learning task.
Temporal Prediction
Prediction of future states in temporal sequences or videos to learn dynamic representations.
Multimodal Learning
Alignment and joint learning between different modalities such as text-image or audio-video.
Knowledge Distillation
Transfer of knowledge from a teacher model to a student model without explicit labels.
Variational Auto-encoders
Learning latent distributions to generate compressed and reconstructive representations.
Video Temporal Coherence
Maintaining coherence of representations between consecutive frames in video sequences.
Learning on Graphs
Generation of proxy tasks on graph structures to learn node and edge representations.
Generative Self-Supervision
Using generative models to create training tasks without external supervision.
Pre-training for Vision
Specialized self-supervised pre-training techniques tailored for computer vision tasks.
Pre-training for NLP
Self-supervised pre-training methods specifically designed for natural language processing.
Learning by Reconstruction
Learning representations by reconstructing the original input from a corrupted or partial version.
Creative Proxy Tasks
Innovative design of label-free training tasks for specific domains.