From Corpora to Co-Evolving Capabilities: Capability-Centric Data Design for Generalist Image Generation

Details

Most large-scale image generation pipelines optimize task-specific datasets in isolation. This paper argues that capability dependencies — text-image grounding before editing, editing before complex transformation — should drive data organization as much as raw scale. Three specialized data engines build complementary supervision: one for text-image grounding, one for inter-image transformation (editing), and one for image-knowledge association. Caption experts align T2I and editing supervision across tasks and granularities. A multi-stage curriculum then jointly evolves task composition, visual-concept distribution, data quality, and image resolution following the dependency order of capability acquisition. Capability-aware evaluation closes the loop through targeted retrieval, expert construction, and gap-aware resampling. At scale: 440M-image T2I corpus, 120M editing pairs, 27M image-entity pairs. These datasets train 3B and 6B multimodal diffusion models from scratch. Evaluation on CPI-Bench with qualitative studies across diverse text-to-image and editing scenarios shows broad visual coverage and effective cross-capability transfer. For AI labs building generalist image generation systems, the main contribution is curriculum design as a first-class engineering concern — data organization and scheduling matter as much as architecture or raw scale. The 17-author team signals substantial research investment behind this infrastructure.