Frontier post-training recipe review with Finbarr Timbers

Details

This podcast episode from Interconnects pairs Nathan Lambert (author of the forthcoming RLHF/post-training book) with Finbarr Timbers (Ai2) for a structured review of how post-training recipes have changed over four years: from InstructGPT's SFT→RM→RL pipeline, through Llama 3 / Tülu 3's SFT→DPO→RL formalization, through DeepSeek R1's large-scale RL emphasis, to the 2026 pattern.\n\nThe central new concept is MOPD — Multi-teacher On-Policy Distillation. Rather than mixing math, code, and agentic RL in a single expensive run (which trades capabilities off against each other), MOPD trains N domain-specialist teachers independently, then trains a general student to minimize reverse-KL against the relevant teacher token-by-token. MiMo Flash V2 introduced this; DeepSeek V4 and Nemotron 3 Ultra scale it to 10+ teachers.\n\nThe hosts also cover 2026 frontier models including GLM 5, Kimi K2.6, and discuss what it would take to bring Olmo-style open recipes to true frontier scale. Career advice in the LLM race closes the episode.\n\nFor practitioners building or studying post-training pipelines, this episode provides one of the most concise structured maps of the current state of the art.