Concept Guidance: Precise, Training-Free Latent Control for Text-to-Image Generation

Details

Controlling specific attributes of generated images — how aesthetically pleasing a result looks, whether hands are correctly formed, whether text renders properly — remains difficult with standard text-to-image diffusion models. Most approaches require either additional training, classifier guidance (needing gradients), or external reference models. Concept Guidance introduces concept-wise mutual information as a lens for understanding which layers in a diffusion model most influence which concepts. The finding is that specific concepts are localized in distinct network layers — a layer-concept specialization that existing methods ignore by treating the full network uniformly. CoG exploits this by quantifying each layer's concept-specific impact and guiding the denoising process using a weighted combination of predictions generated with concept-relevant layers skipped. No fine-tuning, no external models, no reward functions — the method works out-of-the-box on any model that exposes intermediate layer predictions. Demonstrated on PixArt-alpha, SD3, SD3.5, and FLUX.1-dev across multiple concept targets including aesthetics, text rendering, and hand generation. Accepted as an oral presentation at GCPR 2026 (German Conference on Pattern Recognition).