Glossario IA
Il dizionario completo dell'Intelligenza Artificiale
L1 Regularization
Regularization technique that adds a penalty equal to the absolute value of the model coefficients, promoting sparsity and automatically eliminating irrelevant features.
L2 Regularization
Penalization method that adds a term proportional to the square of the coefficients, reducing their magnitude without zeroing them completely to counter overfitting.
Elastic Net
Linear combination of L1 and L2 regularizations that inherits the variable selection properties of Lasso and the stability of Ridge, particularly effective in the presence of correlated features.
Lambda hyperparameter
Regularization parameter controlling the intensity of the penalty applied to coefficients, where lambda=0 corresponds to no regularization and high lambda increases the constraint.
L1 Norm
Vector norm calculated as the sum of the absolute values of the components, used as a penalty term in L1 regularization to induce sparsity.
L2 Norm
Euclidean norm calculated as the square root of the sum of the squares of the components, used in L2 regularization to penalize large coefficients.
Shrinkage
Process of systematically reducing the magnitude of model coefficients toward zero to decrease complexity and improve generalization.
Bias-variance dilemma
Fundamental trade-off in machine learning between reducing bias (systematic error) and reducing variance (sensitivity to data fluctuations).
Alpha coefficient
Mixing parameter in Elastic Net varying between 0 (pure L2 regularization) and 1 (pure L1 regularization) to adjust the relative proportion of the two penalties.
Regularization path
Trajectory of model solutions when the regularization parameter varies, allowing analysis of the evolution of coefficients and their progressive selection.
Weight vector
Set of multiplicative coefficients applied to features in a linear model, whose magnitude is controlled by regularization techniques.
Group penalty
Extension of L1/L2 regularization that penalizes groups of coefficients simultaneously, useful for handling categorical or structured variables.
Adaptive regularization
Variant of regularization where each coefficient receives an individualized penalty based on preliminary estimates, allowing finer variable selection.
Information criterion
Metrics like AIC or BIC that balance model fit and its complexity, often used to select the optimal regularization parameter.
Regularized gradient descent
Optimization algorithm incorporating L1/L2 penalty terms directly into the objective function to efficiently train regularized models.
Coordinate descent
Optimization method particularly efficient for L1 regularization that updates coefficients one by one in an analytical manner.