ForgeWM: Progressive Causal Training for Few-Step Action-Conditioned Video World Models

Details

Interactive video world models must predict game environments in real time — low latency is non-negotiable. Causal (left-to-right) distillation enables few-step video synthesis, but extending it to interactive worlds adds complexity: discrete keyboard states and continuous mouse motion must stay aligned with temporally compressed latent chunks during both training and autoregressive rollout. ForgeWM introduces a four-stage pipeline to transform a bidirectional action-conditioned video generator into efficient few-step world models: domain adaptation, teacher-forced causal training, causal consistency distillation, and on-policy distribution matching with the bidirectional teacher. Budget-specialized students run at steady-state denoising budgets of 1, 2, and 4 steps. Performance on paired Minecraft trajectories shows ForgeWM leads evaluated systems on image quality, reference-aligned motion-profile agreement, action-sign accuracy, and mouse-control accuracy, while achieving the lowest reference LPIPS. The same four-stage recipe transfers to gamepad-controlled FPS gameplay. A dual-path deployment protocol separates latency-critical interaction from optional replay-time refinement — the one-step student re-noises and refines saved drafts at roughly 3x closer quality to the experienced trajectory than regeneration from noise.