Momentum-based Optimization

LAMB

Layer-wise Adaptive Moments optimizer for Batch training that extends LARS by integrating Adam-type adaptive statistics for efficient training of massive models.

← Quay lại