Layer Normalization

Gamma and Beta

Learnable parameters of layer normalization allowing respectively to scale and shift the normalized values to preserve the network's representational power.

← Tillbaka