Layer Normalization

RMS Normalization

Efficient variant of layer normalization using only the root mean square of activations, reducing computational complexity while maintaining performance.

← Wstecz