Where a New Concept Must Enter: Entry Point Gates Cross-Task Usability in Unified Multimodal Models

August 19, 2026 | Source: arXiv AI

Tags: unified multimodal models, transfer learning, concept learning, text-to-image, representation learning

Research on unified multimodal models reveals that where a concept enters the model's computation governs cross-task usability — understanding training installs concepts that can be generated, but generation training cannot produce concepts for generation tasks. A mid-stack alignment objective acquires new visual concepts with 0.1% vs 41% capability degradation.

Details

Unified multimodal models (UMMs) promise that training on both generation and understanding should make each better, but controlled ablations consistently fail to show understanding benefits from generation objectives. This paper investigates why through careful experimental design: novel visual entities (rendered 3D assets with pseudo-words screened for absence from the frozen model) are bound through exactly one task direction, then the other direction is measured.\n\nThe key finding is asymmetry: understanding training installs a concept the model can both match among candidates and actively produce. Generation training installs a concept the model can only match, not produce. The critical variable is not whether both objectives share weights, but where in the model's computation the concept binding occurs.\n\nAn alignment probe predicts this cross-task export across 36 configurations (Spearman ρ = +0.68). The entry layer matters precisely: for a 28-layer model, semantic injection at layer 7 makes concepts drawable, while injections at layer 14+ show no effect. Weight-based editing peaks at layers 10-14. This window only appears where the understanding pathway is a semantic vision encoder — shared weights alone are insufficient without shared semantic format.\n\nExploiting this rule, a mid-stack alignment objective acquires new concepts with just 0.1% relative loss of general text-to-image ability, compared to 41% for the standard generative route.