AI Glossary
The complete dictionary of Artificial Intelligence
Non-Negative Matrix Factorization (NMF)
Linear algebra technique decomposing a non-negative matrix V into two non-negative matrices W and H, such that V ≈ WH, promoting an additive interpretation of data.
Basis Matrix (W)
In NMF, the matrix W contains the basis vectors or 'components' that, when linearly combined, reconstruct the original data, with each column representing a fundamental feature.
Coefficient Matrix (H)
The matrix H in NMF decomposition represents the weights or activation coefficients of each basis (from W) for each data sample, indicating their contribution.
Factorization Rank
Crucial parameter in NMF, the rank (k) determines the number of components or latent factors to extract, controlling the granularity of decomposition and compression level.
Mean Squared Error (Frobenius Norm)
Most common cost function in NMF, calculating the sum of squared differences between elements of V and WH, aiming to minimize the overall Euclidean distance.
Multiplicative Update Rules
Iterative optimization algorithm specific to NMF that updates matrices W and H element by element through multiplication, ensuring maintenance of the non-negativity constraint.
Sparsity Cost
Regularization term added to the NMF cost function to encourage matrices W and/or H to contain many zeros, improving interpretability and feature selection.
Convex NMF
Variant of NMF where the basis matrix W is fixed and pre-defined (often from a data dictionary), making the optimization problem convex and guaranteeing a unique solution for H.
Parallel NMF
Distributed computing approach for NMF, where updates to the elements of matrices W and H are performed simultaneously on multiple cores or computing nodes to accelerate convergence.
Additive Interpretability
Key advantage of NMF compared to other decompositions like PCA, where components are parts that add up to form the whole, facilitating an intuitive understanding of the data.
NMF Initialization
Critical process of choosing initial values for matrices W and H, influencing the convergence speed and quality of the final solution, as the optimization is non-convex.
Orthogonal NMF
Extension of NMF adding an orthogonality constraint on the coefficient matrix H (or the basis W), forcing components to be less correlated and more distinct.
NMF Stability
Measure of the consistency of NMF solutions obtained from different initializations or data subsamples, low stability indicating a potentially non-robust solution.
Co-clustering via NMF
Application of NMF where the simultaneous factorization of rows and columns of a matrix reveals clusters of samples and features sharing common latent structures.
NMF for Signal Processing
Use of NMF to separate audio sources or decompose spectral signals into elementary components (notes, instruments) by exploiting their additive and non-negative nature.
NMF in Text Analysis
Application of NMF to term-document matrices to discover 'themes' (matrix W) and their contribution (matrix H) in each document, offering clear thematic interpretation.
Alternating Coordinates Method (ALS)
Optimization strategy for NMF that solves the problem iteratively by fixing one matrix (H) to optimize the other (W), then reversing the roles until convergence.