Adaptive Learning Rate Methods

Exponential Moving Average (EMA)

Smoothing technique that assigns more weight to recent observations, used in adaptive optimizers to estimate gradient moments.

← Geri