AURA: Unified Multimodal Framework for Conversational Music Editing

Details

Most instruction-guided music editors process each request in isolation, making progressive track refinement cumbersome. AURA treats music editing as a multimodal conversation: a multimodal LLM processes the full dialogue history, an optional image, and reference audio, distilling editing intent into compact concept tokens. A concept-to-audio module injects these tokens alongside frame-aligned reference features into a frozen MusicGen backbone. Critically, only 91M parameters are trained while the 1.9B MusicGen backbone remains frozen — a parameter-efficient design that preserves pre-trained generative quality. Evaluated on Slakh2100 and MoisesDB benchmarks, AURA shows substantial improvements in edit correctness and content preservation versus existing instruction-guided methods, including a 4-5x reduction in FAD (Frechet Audio Distance) for out-of-domain addition and removal tasks. The conversational paradigm opens the door to iterative producer workflows in AI music tools. No production release or user study is described — results are research benchmarks demonstrating proof of concept for the conversational editing architecture.