Adaptive Learning Rate Methods

Bias Correction

Mechanism in Adam that corrects the initial bias of moment estimates towards zero, ensuring unbiased estimates particularly important in the early steps of training.

← Back