Learning 4D Geometric Priors for Inference-Efficient World Action Models

Details

World Action Models for robotic manipulation jointly model visual future dynamics and executable action sequences, but existing video-action co-training methods optimize appearance-oriented latents that can miss the temporally evolving 3D geometry needed for precise manipulation. MECo-WAM addresses this with a training-only solution that adds geometric knowledge without changing the deployed model. During training, MECo-WAM adds a lightweight 4D expert supervised by relational targets from a frozen VGGT encoder alongside the standard video and action experts. Decayed 4D read-mask attention provides restricted geometric guidance early in training and progressively removes this dependency — preventing the action pathway from over-relying on auxiliary geometry signals. Action-aware temporal geometric distillation aligns within-frame geometric relations and their temporal evolution, specifically emphasizing visual regions most relevant to robot actions rather than entire frames. At deployment, all 4D components are stripped away, preserving the original lightweight inference graph. Results: 98.2% success on LIBERO, 92.6% on RoboTwin 2.0, with improvements on real-world manipulation tasks. The co-training approach is architecture-agnostic and can be applied to existing video-action models without redesigning their inference pipelines.