Query-based Submodular Data Selection for Efficient Instruction Tuning

| Source: arXiv AI

Tags: QSGS, Instruction Tuning, Submodular Optimization, Data Selection, Mistral-7B, Llama-3

QSGS selects instruction-tuning data via a submodular graph cut over gradient similarity and fills gaps with synthetic examples, matching or beating full-data performance on Mistral-7B with just 0.5% of the data.

Details

Targeted data selection usually scores each training example against a target set independently, which at small budgets fills the subset with near-duplicates. QSGS (Query-based Submodular functions with Gradient-guided Synthesis) frames selection as a submodular graph cut on a gradient similarity graph: a relevance term pulls examples toward the target task while a dispersion penalty suppresses redundancy. A synthesis step then finds under-covered target directions via residual subspace analysis and generates examples to fill them. Across MMLU, BBH and TydiQA on Llama-2, Llama-3 and Mistral, the authors report QSGS at a 0.5% budget matching or exceeding full-data performance on Mistral-7B, with gains over baselines growing as the budget shrinks.