TRACE-Bench: Decomposing and Diagnosing Multi-Reference Image Generation

August 18, 2026 | Source: arXiv AI

Tags: image generation, multimodal, benchmark, TRACE-Bench, ACM Multimedia

TRACE-Bench evaluates multi-reference image generation across ~1,600 cases using 4 atomic operators, finding that disentanglement is the primary bottleneck — even the best of 9 tested models scores only 0.74 on attribute fidelity. Accepted to ACM Multimedia 2026.

Details

Multi-reference image generation — combining multiple style, subject, or attribute references into one image — lacks rigorous evaluation because existing benchmarks are organized around predefined task types rather than the atomic operations underlying them. TRACE-Bench addresses this with a capability-oriented framework built on four operators: Anchor (f), Disentangle (g), Apply (plus), and Compose (C).\n\nAny multi-reference prompt can be expressed as a formula over these operators. TRACE-Bench includes approximately 1,600 evaluation cases varying complexity from 1 to 8 operator slots, built from 631 formula templates and around 4,000 reference images spanning diverse artistic styles and real-world subjects. The evaluation protocol includes per-capability scoring and a diagnostic tree for recursive failure localization.\n\nTesting 9 leading models reveals a consistent pattern: the bottleneck is not scene-level composition but disentanglement and attribute binding. Even the strongest model achieves only 0.74 on attribute fidelity. The benchmark was accepted to ACM Multimedia 2026.\n\nFor teams building or evaluating multimodal generation systems, TRACE-Bench offers a principled framework to identify where a model fails rather than just measuring aggregate quality.