Improving Influence-based Instruction Tuning Data Selection for Balanced Learning of Diverse Capabilities

Details

Instruction fine-tuning data selection matters: picking the right training examples improves both capability and balanced performance across diverse tasks. Influence-based methods estimate each example's contribution to model predictions and show promise for capability elicitation, but tend to over-select data from high-influence tasks, degrading performance on others—and, counterintuitively, on the high-influence tasks themselves. BIDS addresses this with two steps: first, normalizing influence scores per task to remove the intrinsic advantage high-influence tasks have; second, iteratively selecting the training example with the highest influence on the most underrepresented task at each step. The results are striking: training Llama-3 and Mistral-v0.3 on a 15% subset selected by BIDS outperforms full-dataset training across 7 benchmarks spanning 5 diverse capabilities. BIDS also outperforms both state-of-the-art influence-based algorithms and non-influence-based frameworks. The paper was accepted to EMNLP 2025 (Findings), placing it in a top NLP venue. The practical implication is significant: reducing training data by 85% while improving balanced performance would meaningfully reduce fine-tuning costs.