Deep Optimization

Lookahead Optimizer

Optimization mechanism that periodically updates the 'slow' weights towards the average of 'fast' weights generated by an internal optimizer, improving generalization and convergence stability.

← Tillbaka