Davinci-Tool: Scaling Agentic Tool-Use Supervision with Verifiable Executable Environments

AI summary

Overview

Research area: Agentic machine learning — training data construction and post-training (supervised fine-tuning plus preference optimization) for large language models that perform multi-step tool use in stateful environments.

Technical level: Advanced. The paper assumes familiarity with tool-calling agents, environment state transitions, supervised fine-tuning, and Direct Preference Optimization.

Scope: The paper introduces Davinci-Tool, a 30B tool-use model, and Davinci-ToolData, a 30,395-trajectory execution-grounded dataset built from two complementary pipelines (EGTS and EES) that pair environment coverage with executable outcome verification.

What This Paper Is About

Training reliable tool-using agents requires data that connects a user's request, a multi-step execution, and hard evidence that the intended outcome was actually achieved. Prior work scales either workflow coverage (more tools, more synthetic interactions) or verification (execution checks), but the paper argues these must scale together, because each new environment brings its own states, rules, and dependencies that must also appear in the success criteria. The authors build a unified framework where every training example is grounded in a restorable initial state, an action–observation trace, a resulting environment outcome, and an executable verification result — and then reuse that same infrastructure for both demonstration learning and preference learning.

Key Contributions

1. Davinci-Tool, an execution-grounded training framework. The paper connects verified demonstration construction with outcome-grounded, decision-level preference learning under a shared execution and verification interface W = (S, T, ρ, P, E), where ρ restores the environment to an initial-state snapshot so tasks can be replayed or branched.

2. Davinci-ToolData, a dataset of 30,395 verified tool-use demonstrations spanning 2,260 invoked tool names. EGTS (Execution-Grounded Task Synthesis) builds task-conditioned demonstrations from independently authored systems through exploration and verified replay; EES (Executable Environment Synthesis) constructs and audits executable environments for single-domain (EES-Single) and coordinated cross-domain (EES-Cross) tasks.

3. Strong tool-use performance at the 30B scale. Davinci-Tool-30B improves over its Qwen3-Coder-30B-A3B-Instruct backbone by 35.8, 32.4, and 39.3 percentage points on τ²-bench, BFCL v4 multi-turn, and MCP-Atlas respectively, outperforming Qwen3-Coder-480B-A35B-Instruct on all three benchmarks with 1/16 of its total parameters.

4. Transfer and data-quality evidence. The same Davinci-ToolData supervision substantially improves Qwen3-8B and Qwen3-14B, and under identical backbone and SFT hyperparameters it beats the public Toucan-1.5M release by 7.4, 18.7, and 30.3 points on the three benchmarks despite being a substantially smaller corpus.

Main Findings

• SFT provides the primary capability gains. On Qwen3-Coder-30B-A3B-Instruct, SFT raises τ²-bench from 52.4 to 85.1, BFCL v4 multi-turn from 30.4 to 60.6, and MCP-Atlas from 16.9 to 52.8. Gains appear in both τ² domains and all four BFCL categories.

• Decision-level DPO adds a further, smaller increment. Reusing restorable EES states and executable verifiers yields 1,150 preference pairs, moving the aggregates to 88.2, 62.8, and 56.2 — gains of 3.1, 2.2, and 3.4 points.

• DPO effects are category-dependent. τ² Retail improves by 6.1 points and BFCL Missing-Parameter by 8.0 points, while BFCL Missing-Function decreases by 1.5 points.

• A 30B model beats a 480B reference model on all three aggregates. Davinci-Tool-30B exceeds Qwen3-Coder-480B-A35B-Instruct (τ²-bench average 58.5, BFCL v4 average 32.0, MCP-Atlas 28.1) while using 1/16 as many total parameters. On BFCL v4 its 62.8 score also exceeds Kimi-K2-Instruct, GPT-5.2, and Claude-Sonnet-4.5, but remains below MiniMax-M2.1 and GLM-4.7. On MCP-Atlas its 56.2 exceeds Qwen3-Coder-480B and GLM-4.7 but remains below the other evaluated reference models.

• The supervision transfers to smaller backbones. Qwen3-8B rises from 32.5 to 74.6 on τ²-bench and from 15.4 to 33.9 on BFCL v4; Qwen3-14B rises from 33.4 to 76.8 and from 21.1 to 41.4. MCP-Atlas is omitted for these models because interactions exceed their default 40,960-token context window.

• Corpus size alone does not explain the gains. Under matched backbone and SFT hyperparameters, Davinci-ToolData beats Toucan-1.5M (τ²-bench 77.7, BFCL v4 41.9, MCP-Atlas 22.5) by 7.4, 18.7, and 30.3 points.

• The three data sources are complementary. EGTS gives the strongest single-source BFCL v4 result (57.4); EES-Cross is strongest on τ²-bench (83.4) and MCP-Atlas (47.2). Adding EES-Cross to EGTS+EES-Single improves the three benchmarks by 5.7, 3.0, and 21.3 points, while adding EGTS to EES-Single+EES-Cross yields 2.2, 5.0, and 2.2 points. The full three-source mixture reaches 85.1, 60.6, and 52.8, exceeding the best single-source result by 1.7, 3.2, and 5.6 points.

• Source combinations are not uniformly additive. Some pairwise mixtures underperform a constituent single source, so the authors characterize source complementarity rather than clean marginal effects.

• Corpus composition. EGTS contributes 13,824 trajectories (mean 10,063.9 tokens, 6.04 calls, 350 tool names), EES-Single 8,026 (10,372.7 tokens, 5.87 calls, 1,610 tools), and EES-Cross 8,545 (20,444.7 tokens, 7.01 calls, 1,683 tools). The full corpus has a per-trajectory mean of 13,063.8 tokens and 6.26 calls, with 28,985 distinct normalized requests, 436 declared-tool inventories, 4,672 exposed tool names, and 190,410 tool calls. Median serialized length is 9,284 tokens, with 957 trajectories (3.15%) exceeding 32,768 tokens.

• Environment coverage. EGTS covers 8 authored environment families spanning services, APIs, workspaces, web interaction, and simulation. EES comprises 191 audited domains across 19 categories, 188 of which appear in 246 retained cross-domain compositions — 77 two-domain, 124 three-domain, and 45 four-domain. Of these, 117 span multiple application categories and 129 connect distinct systems within one category.

• Execution-level behavior changes. Paired base/SFT analysis shows recovered τ²-bench tasks have substantially fewer explicit tool errors; BFCL v4 trajectories cover more of the reference call chain and show stronger output-to-argument dependencies; MCP-Atlas loses empty retrievals and malformed terminal outputs despite increased tool interaction.

Methodology in Plain English

The authors treat a tool-use environment as a resettable machine with a state space, a set of callable tools, a reset operation, a transition function, and an evidence space. Every training example must survive an explicit verification call, so the dataset carries not just "what the agent did" but "how we know it worked."

Two pipelines fill this substrate. EGTS handles real, independently authored systems: it first explores the system to discover what outcomes are actually achievable, records evidence of those outcomes, then writes a natural user request around that outcome and replays the task from the same initial state to collect a fresh trajectory. Exploration and demonstration are deliberately separated, so the recorded replay does not leak the exploration path into training — the replay guide is used only during collection, not serialized into SFT examples. Replay is accepted only if the canonical terminal state matches exploration (after stripping volatile fields like timestamps) or, for retrieval tasks, if tool observations and final answers are grounded in the captured evidence.

EES handles synthesized environments instead: it builds rule-governed domains, runs an executor–auditor loop to exercise the tools and check transitions against the rules, and only then generates state-grounded tasks plus executable predicates that define success. A mutation target already satisfied in the initial state is regenerated so success requires an actual interaction-induced change. EES-Cross goes further by composing compatible audited domains through shared entities, roles, policies, and cross-domain state dependencies, using embedding retrieval, an affinity rubric, bounded random walks, and namespace mapping to keep tool names and identifiers distinct.

Training is two-stage. First, supervised fine-tuning on the 30,395 demonstrations, with loss applied only to assistant tokens and tool outputs left in context but excluded from the loss. Second, preference learning: at selected decision points along SFT rollouts in EES scenarios, the same restorable state and context are reused, several alternative decisions are sampled, each branch is continued greedily to termination, and each branch receives a binary verified outcome. Pairs where one branch succeeded and another failed become preference data, and DPO is applied only to the divergent assistant turns — no recursive tree search and no separate process evaluator or expert action generator.

Why This Matters

Impact on research. The paper reframes tool-use data scaling as a joint problem of environment coverage and outcome verification rather than two separate problems. Its distinguishing move is infrastructure reuse: the same restorable states and executable verifiers that accept a demonstration are also what label preference pairs, removing the need for a separate reward model, process evaluator, or search procedure. The demonstration that a 30B model can beat a 480B model on three held-out suites using 1/16 of its total parameters also pushes back on the assumption that agentic tool use requires the largest available backbones.

Real-world applications:

• Coding agents that must select among many tools, read observations, and adapt across multi-step sessions.
• Deep research systems that chain retrieval and synthesis steps and must ground their answers in retrieved evidence.
• Enterprise workflow automation in domains that resemble the evaluated ones, such as retail and telecom account operations, where the order of operations and ownership checks matter.
• Agents operating over heterogeneous MCP tool ecosystems, where tools come from different providers with different names and state conventions.

Industry relevance. The headline economic claim is parameter efficiency — matching or exceeding a 480B model at 30B total parameters with 1/16 the parameter count — which directly affects serving cost and latency. The dataset construction pipelines also target a practical bottleneck: teams with existing internal systems can adapt them to a shared interface and generate verified supervision, rather than hand-labeling trajectories or trusting synthetic data that may not be executable.

Future Directions

• Extending verification to harder outcome types. The framework currently leans on canonical state comparison for mutation tasks, grounded observations for retrieval tasks, and executable predicates for synthesized scenarios. Environments whose success criteria cannot be reduced to these forms are not reported as covered.

• Scaling preference data beyond 1,150 pairs. DPO contributes only 3.1, 2.2, and 3.4 aggregate points, and one BFCL category (Missing-Function) actually decreases by 1.5 points. Whether more forks, more EES scenarios, or different branching policies would increase the DPO yield is left open.

• Closing the remaining benchmark gaps. Davinci-Tool-30B still trails MiniMax-M2.1 and GLM-4.7 on BFCL v4, and trails most evaluated reference models on MCP-Atlas, so the limits of the 30B scale on these suites are unresolved.

• Handling long-context interactions for smaller backbones. MCP-Atlas had to be omitted for Qwen3-8B and Qwen3-14B because interactions exceed their 40,960-token default context window, and 957 corpus trajectories (3.15%) exceed 32,768 tokens before training-time handling — an open question for how far down the model scale this supervision can be pushed.

Target Audience

Researchers and engineers working on LLM agents, tool use, and post-training data pipelines will get the most from this paper, particularly those interested in how to make training supervision verifiable rather than merely plausible. It is also relevant to practitioners building MCP-based or enterprise workflow agents who need to decide whether to invest in execution infrastructure for data generation, and to anyone studying parameter-efficient alternatives to frontier-scale models for agentic tasks. Readers without background in reinforcement learning from preferences or environment-based agent evaluation will find the methodology sections dense; the overview and results tables are accessible without that background.

Authors’ abstract

Reliable agentic tool use requires training data that pair diverse multi-step workflows with executable evidence that the requested outcomes were achieved. Scaling either trajectory coverage or verification alone is insufficient as new environments introduce distinct states, rules, and dependencies that must also be reflected in the success criteria. To address this challenge, we introduce Davinci-Tool, a 30B tool-use model, and Davinci-ToolData, an execution-grounded dataset designed to scale environment coverage and outcome verification together. Davinci-ToolData preserves restorable states, execution traces, outcomes, and verifiers as a unified substrate for supervision. Two complementary pipelines populate this substrate: Execution-Grounded Task Synthesis (EGTS) derives verified demonstrations from independently authored systems, and Executable Environment Synthesis (EES) synthesizes auditable environments, state-grounded tasks, and executable verifiers across single and coordinated domains. These pipelines yield 30,395 verified trajectories spanning 2,260 invoked tool names. We first train on these demonstrations with supervised fine-tuning, then reuse EES states and verifiers to compare alternative actions and construct outcome-grounded preferences for Direct Preference Optimization. Davinci-Tool improves Qwen3-Coder-30B-A3B-Instruct from 52.4 to 88.2 on $τ^2$-bench, from 30.4 to 62.8 on BFCL v4 multi-turn, and from 16.9 to 56.2 on MCP-Atlas, outperforming Qwen3-Coder-480B-A35B-Instruct on all three benchmarks with only 1/16 of its total parameters. Code is available at https://github.com/MassLab-SII/Davinci-Tool.