Batch Normalization

Fact

A technique that normalizes neuron signals in each training batch.

In Plain Words

Batch Normalization is like a factory resetting every part to the same baseline. No bolt starts too long or too short before assembly.

It evens out neuron signals in each training batch. Deep nets use it to learn faster and wobble less.

Related Concepts

• Layer Normalization
• Gradient Descent
• Convolutional Neural Network
• Residual Network

Layer Normalization
Both steady training, but Batch Normalization uses batch stats while Layer Normalization works per sample.

Gradient Descent
Batch Normalization can reduce training wobble, so Gradient Descent can settle more easily.

CNN
Batch Normalization was widely used in CNNs to make training faster and steadier.

ResNet
Batch Normalization often appears inside deep ResNet models.