BERT Architecture

Pre-training

Unsupervised training phase on large corpora where BERT learns general linguistic representations via MLM and NSP. This step establishes the knowledge foundations of the model before task-specific fine-tuning.

← Geri