mlm-bridge-training

Installation
SKILL.md

MLM vs Bridge Training

For how they differ, the arg mapping tables, gotchas, and translation script, see:

  • docs/megatron-lm-to-megatron-bridge.md

Correlation Testing

Use vanilla_gpt_pretrain_config for loss-correlation testing. This recipe uses bare GPTModelProvider defaults (LayerNorm, GeLU, learned_absolute position embeddings, vocab_size inherited from tokenizer) — matching MLM pretrain_gpt.py defaults with no args.

MLM Correlation Run (2L/256H, 1 GPU)

Installs
1
GitHub Stars
822
First Seen
Apr 19, 2026
mlm-bridge-training — nvidia-nemo/megatron-bridge