DiffusionGemma: 4x faster text generation
| Source: Google DeepMind Blog
Tags: DiffusionGemma, Google DeepMind, Gemma 4, text diffusion, open weights, inference speed, MoE
Google DeepMind released DiffusionGemma under Apache 2.0 — a 26B MoE model that generates 256 tokens in parallel using text diffusion, hitting 1,000+ tokens/sec on H100 and 700+ on RTX 5090, up to 4x faster than autoregressive models, with immediate Hugging Face, vLLM, and Unsloth support.
Details
DiffusionGemma is an experimental open model from Google DeepMind that challenges the dominant autoregressive paradigm. Instead of generating one token at a time, it starts from noise and refines a full block of 256 tokens simultaneously using a diffusion head built on the Gemma 4 26B MoE architecture.\n\nThe practical result: 1,000+ tokens/sec on a single NVIDIA H100, 700+ tokens/sec on an RTX 5090, and 150 tokens/sec on DGX Spark — speeds that make interactive local AI workloads feasible for the first time. At 3.8B active parameters (from a 26B total), it fits under 18GB VRAM when quantized, accessible on high-end consumer hardware.\n\nThe key tradeoff is output quality. Google DeepMind explicitly states DiffusionGemma's quality is lower than standard Gemma 4 and recommends the autoregressive version for production. The model is designed for latency-sensitive single-user scenarios: inline editing, code infilling, non-linear text structures, amino acid sequences, and agentic loops where speed matters more than peak quality.\n\nReleased under Apache 2.0 with day-zero Hugging Face Transformers, vLLM, and Unsloth support. Bi-directional attention across the 256-token block enables code infilling and non-linear structures that autoregressive models handle poorly.