GenDSR: Transferring Representations from Video Generation Models for Dynamic Spatial Reasoning
| Source: arXiv AI
Tags: GenDSR, DSR-Bench, video generation models, vision-language models, spatial reasoning
GenDSR feeds intermediate features from video generation models into vision-language models for dynamic spatial reasoning, reaching a state-of-the-art 67.2% on DSR-Bench.
Details
Vision-language models struggle to keep track of scene state while spatial relations change over time. Video generation models, trained to synthesize coherent motion, carry useful spatiotemporal representations. GenDSR transfers intermediate video-generation features into a VLM through two branches: a raw branch that keeps holistic generative context and a delta branch that emphasizes temporal change. Task-conditioned tokenwise gates fuse the two and inject the result into the VLM's native visual representation. On DSR-Bench, GenDSR scores 67.2%, which the authors report as a new state of the art. The result supports a growing pattern of reusing generative video models as feature extractors for perception and reasoning tasks.