AI Glossary
The complete dictionary of Artificial Intelligence
SVD (Singular Value Decomposition)
Fundamental method decomposing any matrix M into the product UΣVᵀ where U and V are orthogonal and Σ is diagonal with singular values. Provides the best rank-k approximation in the least squares sense and reveals the intrinsic structure of the data.
Truncated Singular Value Decomposition
Variant of SVD retaining only the k largest singular values and associated vectors to explicitly reduce dimensionality. Optimizes the bias-variance trade-off by eliminating noise while preserving the main components of information.
LU Decomposition
Factorization of a square matrix into the product of a lower triangular matrix L and an upper triangular matrix U. Fundamental for efficiently solving systems of linear equations and calculating determinants.
QR Decomposition
Decomposition of a matrix into the product of an orthogonal matrix Q and an upper triangular matrix R. Essential for solving least squares problems and implementing numerically stable algorithms.
PARAFAC Factorization
Extension of matrix factorization to higher-order tensors using a parallel factors decomposition. Captures multi-dimensional interactions in tensor data for applications such as multi-sensor signal analysis.
Tucker Decomposition
Generalization of SVD to tensors decomposing a tensor into a core tensor and factor matrices for each mode. Offers greater flexibility than PARAFAC by allowing different ranks for each dimension.
ALS (Alternating Least Squares) Factorization
Iterative algorithm alternately optimizing each matrix factor while fixing the others. Efficiently converges to local solutions and constitutes the reference method for large-scale recommendation systems.
Low-Rank Optimization
Optimization problem seeking the best matrix approximation under minimal rank constraint. Fundamental for data compression, denoising, and structure extraction in high-dimensional data.
Matrix Completion
Task of imputing missing values in a matrix by assuming an underlying low-rank structure. Key applications in recommendation systems and reconstruction of partially observed data.
Rank-k Approximation Reconstruction
Eckart-Young theorem guaranteeing that SVD truncation provides the best rank-k approximation in terms of Frobenius norm. Theoretically establishes the optimality of dimension reduction methods based on SVD.
Probabilistic Matrix Factorization
Bayesian approach treating latent factors as random variables with prior distributions. Naturally allows for regularization incorporation and quantifies uncertainty in predictions.
Stochastic SVD Decomposition
Randomized algorithm computing an SVD approximation using random projections to reduce computational complexity. Particularly effective for massive matrices where exact SVD is prohibitively expensive.
Diagonal Block Factorization
Specialized decomposition exploiting a block structure to parallelize computation and reduce memory complexity. Essential for distributed processing of large-scale structured matrices.
Independent Component Decomposition
Factorization separating a multivariate signal into statistically independent components maximizing non-Gaussianity. Fundamental for signal processing and blind source separation.
Outlier-Robust Factorization
Matrix factorization variant resistant to corrupted observations or anomalies using robust norms. Critical for real-world data often contaminated by noise or measurement errors.