What Remains in Sight? Autoregressive Video Decoding as Representation-Guided Context Rewriting

| Source: arXiv AI

Tags: ReCR, Autoregressive Video Generation, KV Cache, Text-to-Video, Long-Horizon Video

ReCR is a training-free method that treats which past frames an autoregressive video model sees as an explicit decision, improving long-horizon generation across several backbones at matched token budgets.

Details

Long video generation increasingly uses autoregressive rollout, but inference pipelines usually choose the visible past implicitly through recent windows, compressed caches or external memory. ReCR, from authors including Leonidas Guibas, uses internal decoder representations to pick context units that match the global rollout state, avoid redundancy with recent output, preserve temporal boundaries and widen spatial layouts in view. Selected units are mapped onto a compact causal axis before decoding the next block. The authors report consistent gains on long-horizon text-to-video benchmarks across multiple autoregressive backbones under matched visible-token budgets. The abstract does not list specific metrics.