The Pulse
Google’s ToolGrad Cuts Agent-Training Data Costs
Google Research introduces ToolGrad, a framework that generates verified API workflows before writing user prompts. The system reaches a 99.8% pass rate while producing more complex tool-use chains with fewer tool-execution steps.
AI.info Team ·
ToolGrad reaches a 99.8% pass rate when generating training examples for AI systems that use external tools, according to Google Research. The framework also produces longer tool-use chains while reducing the number of tool-execution steps required to create each example.
Google Research introduced ToolGrad on September 10, 2026, describing it as an answer-first method for building datasets. Rather than inventing a user request and then asking an agent to search for a workable API sequence, ToolGrad creates a valid sequence of tool calls first and generates a matching request afterward.
The method targets one of the more expensive parts of training an agent: producing reliable examples that connect a natural-language request to a sequence of API calls and a final response. Google’s post presents the work alongside a research paper and an accompanying implementation.
ToolGrad Reverses the Dataset Pipeline
Earlier query-first systems begin by sampling APIs and creating a hypothetical instruction. An agent then searches for a solution, often through depth-first search, and keeps the successful traces for training. Google says that process wastes model calls on requests that cannot be solved by the selected tools and can allow failed or incorrect steps into the resulting dataset.
ToolGrad starts with at least one successful API action. Its workflow then grows through repeated rounds in which candidate APIs are proposed, executed in parallel, evaluated, and either selected or discarded. Once the chain is complete, a language model updates the synthetic user request and final answer so they match the verified workflow.
Four Modules Build a Verified Tool Chain
The system divides the work among four components. An API Proposer narrows a sampled group of APIs to candidates that could extend the current workflow. API Executors test those candidates and produce execution reports, while an API Selector chooses the strongest next call.
An LLM Updater then revises the synthetic request and response around the expanded API sequence. Google compares the selector’s feedback to a textual gradient: instead of a numerical signal changing model weights, plain-language evaluation directs the next step in constructing the workflow.
Each completed sample contains a user prompt, a verified tool-use chain, and an AI response. The design is intended to make the tool sequence the dependable part of the example before the system asks a model to express the user’s goal in natural language.
99.8% Pass Rate Across 16,000 APIs
Google tested ToolGrad against the query-first process used with ToolBench, an API database containing more than 16,000 real-world APIs. ToolGrad produced an average of 3.4 ground-truth tool uses per sample, compared with 2.1 for the depth-first-search baseline.
The pass rate rose from 63.8% for the baseline to 99.8% for ToolGrad. The paper reports that average language-model invocations fell slightly, from 64.5 to 63.9, while tool-use steps dropped from 34.3 to 20.0. Google attributes the remaining 0.2% of failures to cases in which the system could not obtain a successful response from selected APIs across its iterations.
The result is not a claim that every part of agent training becomes cheaper. It measures the cost of generating the synthetic tool-use data, and the comparison uses the particular ToolBench setup described by the researchers.
Gemma-3 Models Improve on Unseen Tools
For a second experiment, the researchers created ToolGrad-500, a 500-sample dataset built from ToolBench APIs. They fine-tuned Gemma-3 models with 1 billion, 4 billion, and 12 billion parameters, naming the resulting systems ToolGrad-1B, ToolGrad-4B, and ToolGrad-12B.
On the Berkeley Function Calling Leaderboard, which uses a different tool set from ToolBench, all three fine-tuned models improved on their base versions. ToolGrad-12B scored 83.1, close to Gemini 2.5 Pro at 83.2 and Claude 4.5 Opus at 82.8 in the comparison reported by Google. GPT-5 scored 74.4 in the same table.
Google says the 500-example dataset was generated with Gemini 2.5 Flash-Lite, yet Gemma-3-12B fine-tuned on that data exceeded the teacher model in the reported evaluations. The result suggests that the structure of the examples, rather than only the size or capability of the model that generated them, affects tool-use performance.
ToolGrad’s Scaling Limits Are Already Visible
The paper identifies limits alongside the headline results. In a scaling study, pass rates tended to level off between eight and twelve iterations, and the researchers selected ten iterations for ToolGrad-500 as a cost-conscious setting. The system also began generating similar tool-use patterns across samples because each example was created independently without a shared memory mechanism.
Google also cautions that synthetic requests may not reflect how people naturally describe their goals. The current work focuses on supervised fine-tuning rather than reinforcement learning, and the authors say it does not yet show how well the data transfers to every agent framework or reasoning method.
ToolGrad therefore offers a narrower result than a general solution to agent training: it makes one data-generation pipeline more successful and less wasteful in the experiments reported. Its next technical test is whether the answer-first method can keep producing varied, human-aligned workflows after the current iteration and diversity limits appear at larger scale.