AI Glossary
The complete dictionary of Artificial Intelligence
Online Matrix Factorization (OMF)
Set of techniques decomposing a matrix into lower-rank factors sequentially, updating the factors as new data arrives, without requiring complete retraining.
Stochastic Gradient Descent (SGD) for OMF
Iterative optimization algorithm that updates matrix factors using the gradient of the loss function computed on a single sample (or mini-batch) at a time, ideal for data streams.
Incremental SVD
Variant of Singular Value Decomposition that updates existing singular vectors and singular values to incorporate new columns or rows of data without recalculating from scratch.
Projection Approximate Subspace Tracking (PAST)
Recursive algorithm that tracks the subspace spanned by the dominant eigenvectors of a covariance matrix in real-time, minimizing a least squares cost function.
Exponential Forgetting
Mechanism giving more weight to recent observations than to older ones in the update process, allowing the model to adapt to changes in data distribution (concept drift).
Streaming Recommender Systems
Recommendation systems that use online matrix factorization to continuously update user and item profiles from new interactions, ensuring relevant suggestions in real-time.
Recursive Least Squares (RLS) Method
Adaptation algorithm that minimizes a weighted least squares cost function recursively, offering fast convergence at the cost of higher computational complexity than SGD.
Block Update
Strategy where matrix factors are not updated for each new data point, but after accumulating a block of observations, offering a compromise between responsiveness and computational efficiency.
Online Non-negative Matrix Factorization (NMF)
Online variant of matrix factorization that imposes non-negativity constraints on the factors, producing additive and interpretable decompositions, often used for text or image analysis.
Online Loss Function
Error measure calculated on new observations to guide the update of factors, typically mean squared error or a divergence (e.g., KL-divergence for count data).
Robustness to Outliers
Ability of an online factorization algorithm to not be significantly degraded by the presence of noise or erroneous observations in the data stream, often through robust loss functions.
Per-Sample Complexity
Measure of the computational cost (time and memory) required to update matrix factors with a single new observation, a key criterion for evaluating the scalability of online algorithms.
Cold-Start
Challenge in online factorization where the model must provide predictions before having accumulated enough data to reliably estimate latent factors for new users or items.
Oja's Algorithm
Simple stochastic algorithm for the online computation of the principal eigenvector of a covariance matrix, fundamental for real-time subspace tracking.