Accelerating Text-to-Video Generation with Calibrated Sparse Attention
| Source: Apple ML Research
Tags: Apple, sparse-attention, video-generation, diffusion-models, Wan-2.1, inference-optimization, ECCV
Apple researchers introduce CalibAtt, a training-free sparse attention method that speeds up video diffusion models by up to 1.58x on Wan 2.1 14B and Mochi 1 by identifying and skipping token-pair connections that consistently produce negligible attention scores.
Details
Apple ML researchers present CalibAtt (Calibrated Sparse Attention), a training-free inference acceleration method for text-to-video diffusion models, accepted to ECCV 2026. The core insight: a significant fraction of token-to-token attention connections consistently yield negligible scores across different inputs, and their patterns often repeat across queries — meaning they can be safely skipped without degrading output quality. CalibAtt runs an offline calibration pass over representative inputs to identify block-level sparsity and repetition patterns stable enough to generalize. These patterns are compiled into optimized attention kernels per layer, attention head, and diffusion timestep. At inference, only selected connections are computed densely; the rest are skipped in a hardware-efficient manner, converting theoretical FLOP savings into real latency reductions. Experiments on Wan 2.1 14B, Mochi 1, and few-step distilled models at multiple resolutions show up to 1.58x end-to-end speedup while maintaining video quality and text-video alignment scores. Because the method requires no retraining, it can be applied to already-deployed models — a low adoption barrier that distinguishes it from distillation or quantization approaches that require fine-tuning.