Beyond Next-Token Prediction: A Performance Characterization of Diffusion versus Autoregressive Language Models
| Source: Apple ML Research
Tags: diffusion language models, autoregressive models, inference efficiency, Apple, UC Berkeley, LLM architecture
A UC Berkeley / Apple study finds diffusion language models achieve higher arithmetic intensity than autoregressive models but fail to scale to long contexts — and that ARMs outperform DLMs on batched inference throughput. The key fix for DLMs: fewer sampling steps.
Details
This paper provides the most comprehensive performance characterization to date comparing Diffusion Language Models (DLMs) and Autoregressive Language Models (ARMs), combining theoretical analysis with empirical profiling across computation, memory, and latency dimensions. Key findings: DLMs achieve higher arithmetic intensity than ARMs because they process all token positions in parallel, but this advantage collapses at longer contexts because DLMs fail to scale efficiently there. Block-wise decoding for DLMs partially restores this by decoupling arithmetic intensity from sequence length. Meanwhile, ARMs demonstrate superior batched inference throughput because they benefit more from cross-sequence parallelism. The practical implication for open-source DLMs: the primary bottleneck is sampling efficiency, not model quality. Reducing the number of inference steps is the critical path to making DLMs competitive with ARMs on latency — current DLMs require too many forward passes. This work is timely: interest in diffusion-based language models has grown significantly in 2025-2026, and the field needed an honest performance characterization before DLMs could be seriously evaluated for production deployment.