Separating Common and Unique Directions for Data-Free Model Merging
| Source: arXiv AI
Tags: model merging, task arithmetic, CUSD, NeurIPS 2026, fine-tuning
CUSD, a data-free pre-merging step that splits task vectors into shared and task-specific directions, improved 42 of 44 model-operator combinations and gained up to 22.5% over standard arithmetic model merging.
Details
Arithmetic model merging combines fine-tuned models without training, but it applies one element-wise rule to directions that differ in how well they merge. The authors find a consistent two-regime structure in a shared directional basis: a low-dispersion common bulk shared across tasks and a high-dispersion, task-dominated tail carrying disproportionate energy. Common-Unique Subspace Decomposition (CUSD) merges the common components with an existing arithmetic operator and keeps the unique residuals separately. Across 11 base models, four model families and four merging operators, CUSD improved 42 of 44 combinations, with gains up to 22.5%. The paper is accepted at NeurIPS 2026 and code is public.