Słownik AI
Kompletny słownik sztucznej inteligencji
Streaming Convex Optimization
Optimization paradigm where data arrives as a continuous stream and decisions must be made in real-time without storing all observations.
Mirror Descent Algorithm
Generalization of gradient descent using a Bregman divergence function to perform updates in the dual space.
Projection Operator
Mathematical operation that projects a point onto a convex set, essential for maintaining constraints in online optimization.
Follow-the-Regularized-Leader
Online algorithm that solves a regularized optimization problem at each round based on cumulative losses up to that point.
Adagrad Algorithm
Adaptive optimizer that adjusts the learning rate for each parameter based on the sum of squares of historical gradients.
RMSprop Optimization
Algorithm that maintains an exponential moving average of squared gradients to normalize parameter updates.
Convex Loss Function
Convex loss function guaranteeing the existence of a unique global optimum and convergence of optimization algorithms.
Subgradient Method
Extension of gradient descent for non-differentiable functions using subgradients to guide optimization.
Dual Averaging
Online optimization technique that maintains an average of gradients in the dual space before projecting into the primal space.
Proximal Gradient Method
Algorithm combining gradient descent and proximal operator to handle non-differentiable regularization terms.
Mini-batch Optimization
Intermediate approach between SGD and full batch processing, using small groups of observations to estimate gradients.
Convergence Rate Analysis
Theoretical study of the speed at which online optimization algorithms converge to their optimal solution.