GPT‑OSS‑120B: Choose the Training Regime Before Hyperparameters

These are engineering starting points, not an official OpenAI CPT recipe. Choose a stage, then calculate token batch from the actual data-parallel degree.

Recommended starting point
Global token batch calculator
tokens/update = DP ranks × microbatch × sequence length × grad accumulation
Harmony prompt contract (SFT / agent stages)
system   Reasoning: medium  |  Valid channels: analysis, commentary, final
developer   Domain policy, output constraints, and tool definitions
user   User request
assistant / analysis   Reasoning for the current turn (train only high-quality traces)
assistant / commentary   Tool call or user-visible intermediate message
assistant / final   Final user-visible answer
In multi-turn training, retain historical finals but do not put old reasoning traces back into the next-turn context. Raw CPT text and code do not need to be converted into Harmony conversations.
Total GPU count is not DP ranks: under TP/EP/FSDP, use the actual data-parallel replicas. CPT at 1–4M tokens/update and SFT at 16–128 sequences/update are different optimization regimes.