Scaling Laws

Critical Batch Size

Optimal batch size beyond which further increase no longer produces significant improvements in training speed.

← Geri