Google Research Releases ToolGrad: Answer-First Framework Hits 99.8% Pass Rate for Tool-Use Data Generation

Details

Training LLMs to reliably call external tools requires high-quality datasets pairing user queries with correct API execution chains. The prevailing query-first approach—used by ToolBench and ToolACE—generates a user query first, then runs a depth-first search (DFS) agent to find a valid tool path. That search frequently dead-ends, wasting compute and discarding up to 36% of generation attempts.\n\nToolGrad, from Google Research in collaboration with the University of Tokyo, RIKEN AIP, and Tohoku University, reverses this order. It runs a four-module loop: an API Proposer narrows candidate APIs, API Executors run them in parallel and produce execution reports, an API Selector picks the best call using textual gradients, and an LLM Updater rewrites the query to match the verified chain. The result is a confirmed working API execution path before any query is written—reducing generation failures to 0.2%.\n\nThe practical gains are measurable. On the ToolBench database (16,000+ real-world APIs), pass rate jumped from 63.8% to 99.8%, chain length increased from 2.1 to 3.4 tool uses per sample, and total steps per sample fell 42% from 34.3 to 20.0. Gemma-3 1B, 4B, and 12B models fine-tuned on only 500 ToolGrad samples score 83.1 on the Berkeley Function Calling Leaderboard—matching much larger proprietary models.\n\nAll artifacts are Apache-2.0 licensed: the ToolGrad-500 dataset, three fine-tuned Gemma-3 checkpoints, and a PyPI package. Teams building tool-calling pipelines or fine-tuning for function calling can adopt this directly.