Beyond Visual CoT: Internalized Visual Thinking for Proactive Video Reasoning

Details

Visual chain-of-thought (Visual CoT) improves multimodal reasoning by generating intermediate reasoning images, but this comes at significant inference cost — a dealbreaker for real-time video applications. IVT (Internalized Visual Thinking) reframes this: instead of generating pixels at inference, the model learns to predict latent representations of future frames during training. The training objective jointly optimizes textual prediction and next-embedding prediction on unlabeled video. At inference, IVT generates the answer directly — no future frame synthesis or re-encoding. The result: comparable or better performance versus explicit Visual CoT on six evaluation settings, with average end-to-end latency reduced by more than 5x. This is an important efficiency result for practitioners deploying multimodal video models. The finding suggests that explicit pixel-space generation at inference may be unnecessary for video reasoning — the model can internalize visual foresight during training. The paper performs controlled studies across representation choices, decoder designs, prediction horizons, and data mixtures, making the findings robust.