Rethinking Token-wise Feature Caching: Accelerating Diffusion Transformers with Dual Feature Caching
| Source: arXiv AI
Tags: diffusion-models, FLUX, inference-optimization, image-generation, video-generation, DiT
DuCa challenges the assumption that 'important tokens' must always be computed in diffusion transformers—finding that consistent computation of selected tokens is unnecessary and sometimes worse than random selection—then introduces alternating aggressive/conservative caching with random token selection, published in IEEE Transactions on Image Processing.
Details
Feature caching accelerates diffusion transformers by reusing cached features from prior timesteps instead of recomputing them. Token-wise caching methods further differentiate by computing only 'important' tokens each step. DuCa systematically questions whether this selectivity actually helps. The paper poses two questions: Is it necessary to always compute the 'important' tokens? Are those tokens actually important? The answers are surprising: consistently computing selected tokens is not necessary, and the selection criteria are often ineffective—sometimes performing worse than random selection. Based on these findings, DuCa introduces a dual strategy that alternates between aggressive caching (caching most features) and conservative caching (caching fewer features) iteratively, while selecting tokens for computation randomly rather than by importance. This simplifies the approach while improving results. Experiments demonstrate significant improvements over previous token-wise caching methods on DiT, PixArt, FLUX, and OpenSora—covering both image and video generation. The work is published in IEEE Transactions on Image Processing (2026), vol. 35.