What Makes Good Agentic Data? An ACE Lens on Data Generation for LLM Agents

AI summary

Overview

Research area: Data generation for large language model (LLM) agents — how interaction data connecting environments, tasks, trajectories, and success signals are constructed, verified, and selected for agent training and evaluation.

Technical level: Advanced. The paper is a cross-domain synthesis that introduces formal notation (a POMDP interaction model, a factorized data object, and a constrained optimization objective), though its organizing ideas are explained in accessible terms.

Scope: The paper proposes a two-level framework — a common factorized data object (E, q, τ, v) and the Accuracy–Complexity–divErsity (ACE) lens — to unify and compare agentic data generation pipelines that are currently described in domain-specific ways.

What This Paper Is About

LLM agents learn to act by invoking tools, operating systems, browsing, editing files, and interacting with simulated or physical worlds, which requires experience that links decisions to observations and state changes. Such interaction data are expensive to collect by hand and hard to verify and expand, so automatic agentic data generation has become central to scaling agent training and evaluation. The paper argues that the literature is fragmented because it is organized by application domain (API calls, repository tasks, GUI demonstrations, simulator rollouts, scientific discovery) rather than by generation mechanism, which conflates how candidates are constructed with how they are verified and selected. Its goal is to supply a common account of what a pipeline builds and how the resulting data distribution should be shaped.

Key Contributions

1. An ACE-centered synthesis of agentic data generation. The authors organize prior work around how generation pipelines ensure Accuracy, construct and calibrate learner-relative Complexity, and expand effective Diversity, together with domain-specific evidence, measurement choices, costs, and limitations.

2. A cross-domain formulation of agentic data. They define a common factorized data object (E, q, τ, v) that makes the generated components and their consistency relations explicit across tool use, software engineering, GUI, embodied, social, and scientific agents.

3. A mechanism-oriented taxonomy of generation paradigms. Pipelines are organized by their primary anchor and dependency structure rather than by domain labels, separating how data are constructed from how generated candidates are assessed and selected.

4. ACE as a constrained distribution-design objective. The paper formalizes generation as maximizing learner-relative complexity utility and batch-level diversity subject to an accuracy acceptance constraint, making explicit that accuracy establishes the feasible set while complexity and diversity only shape which valid data are most useful.

Main Findings

• A common data object underlies heterogeneous domains. Agentic data is decomposed as d = (E, q, τ, v): an environment specification, a task signal, an interaction realization, and an optional verifier or reward interface. The first three factors define the interactive problem and its realization; v records how consistency or outcome is evaluated and may be a schema checker, executable test, terminal-state predicate, policy rule, proof assistant, LLM judge, or a hybrid.

• Generation pipelines split into forward and reverse families. Forward generation follows the dependency E → q → τ, grounding tasks and interactions in an existing environment. Reverse generation alters which artifact anchors the others and includes task-first pipelines (constructing environments around a target capability or goal), trajectory-first pipelines (recovering tasks from observed valid behavior), and structure-first pipelines (using an intermediate tool graph, function path, dialogue skeleton, or task blueprint). Adaptive and self-evolving systems are treated as a cross-cutting extension.

• Environments come from three overlapping sources. Real or curated specifications crawled from APIs, MCP services, repositories, websites, and applications; LLM-synthesized tools, rules, and domain descriptions; and programmatic construction that explicitly implements databases, transition dynamics, simulators, and validators. The third route offers stronger support for stateful interaction, controlled reset, and reliable verification, but at higher engineering cost.

• Accuracy is an admission condition, not a compensable benefit. The validity decision is conjunctive over environment, task, interaction, and verifier: A(d) = V_E(E) ∧ V_q(q|E) ∧ V_τ(τ|E,q) ∧ V_v(v|E,q,τ). A correct-looking trajectory does not compensate for an infeasible task, and a correct terminal state does not compensate for a verifier that accepts policy-violating shortcuts. Difficulty or variation cannot compensate for invalidity.

• Accuracy has four factor-level targets. Environment accuracy concerns consistent tools, states, policies, and transitions; task accuracy concerns interpretability and feasibility under the relevant state, tools, and policies; interaction accuracy concerns schema-compliant actions, causally grounded observations, and policy-consistent behavior; and verifier accuracy concerns correctly recognizing success, rejecting meaningful failures, and avoiding unintended incentives.

• Interaction accuracy deserves particular emphasis. Unlike environment and task accuracy, which affect learning through the conditions they impose, the trajectory directly provides the behavioral supervision the agent learns from, so errors in τ can be directly imitated. Much agentic data validation therefore focuses on trajectory-level filtering of structurally invalid, semantically inconsistent, or execution-inconsistent interactions.

• Complexity should be calibrated, not maximized. The formulation uses C_z(d) for the difficulty of an instance under a declared learner and execution configuration z (model, available scaffold and tools, verifier, and inference budget), with a utility g_z(C_z(d)) that commonly favors a learnable band near the current model frontier.

• Diversity is a batch-level property. D(B_A) rewards coverage and non-redundancy over environments, tasks, and interaction realizations, rather than surface variation or dataset size.

• The literature shows three coherent shifts. From plausibility judgments toward execution-grounded accuracy; from static difficulty heuristics toward model- and configuration-relative complexity; and from surface variation toward behavioral coverage. Agentic data are also expanding beyond fixed post-training trajectories toward pre- and mid-training supervision and closed-loop experience that changes with the learner.

• No quantitative benchmarks are reported in the available content. The provided text describes the framework, taxonomy, and tables of representative methods (Tables 1 and 2); it does not report dataset sizes, performance numbers, or experimental results.

Methodology in Plain English

The authors treat this as a conceptual and analytical contribution rather than an experimental one. Their approach has four moves.

First, they define what an agentic interaction actually is by adopting an existing partially observable Markov decision process formulation, M = (U, S, A, O, P, R), where the policy acts on observable history rather than latent state. This lets them cover tool use, web and GUI agents, coding agents, and embodied settings under one abstraction, and it makes a distinction they stress: a trajectory records what the policy observes and does, while an executable environment may additionally maintain latent state for transition and reward computation.

Second, they parameterize a concrete environment as e = (D, F, P_rule, Ω, v) — an optional state carrier, an available tool or action set, policies and domain constraints, an observation interface, and an optional success interface — and use this to argue that E ranges from a static interface specification to a complete interaction substrate.

Third, they organize existing pipelines by which factor anchors the others and how the remaining factors are instantiated, rather than by domain label. This yields the forward versus reverse split and the finer task-first, trajectory-first, and structure-first groupings.

Fourth, they cast generation as constrained distribution design: maximize a weighted combination of complexity utility over the valid batch and batch-level diversity, subject to the probability that the validity check passes being at least a required acceptance level. If no candidate passes the accuracy gate, the utility is defined as zero. They then walk through the literature along the three ACE dimensions to show which mechanisms serve which requirement, including the interactions and costs among them. They explicitly note that ACE is an analytical lens rather than an exhaustive checklist for dataset governance — cost, efficiency, and safety remain important constraints — and that it is not a partition of the literature into three disjoint method families, since a single mechanism (such as a verified blueprint) can improve accuracy, lengthen dependency structure, and enable controlled recombination simultaneously.

Why This Matters

Impact on research. The paper addresses a structural problem in a fast-moving field: similar mechanisms appear under different terminology across domains, and data produced by substantially different processes are reported under heterogeneous, difficult-to-align criteria. By separating the architecture of data generation from the objective used to evaluate and allocate its outputs, it provides vocabulary for comparing pipelines across domains and for reasoning about why a dataset that passes validity checks may still be useless for learning. The framework also reframes the central challenge: not generating more data, but continually allocating valid, informative, and non-redundant experience as agents and environments evolve.

Real-world applications. The domains the framework is explicitly built to cover include:

• Tool-using agents, where actions are structured API calls and observations are tool returns, including MCP-based environments.
• Software engineering agents, where the environment includes a repository, shell, dependencies, and tests, and validation uses repository setup, compilation, and test execution.
• GUI, web, and mobile agents, where actions are interface operations and observations are visual or structured page states, and validity can be inspected through application or simulator state.
• Embodied and scientific agents, where the environment may be a simulator, a physical world, or a scientific discovery setting, with formal-reasoning agents validated via proof assistants.

Industry relevance. The authors are affiliated with Huawei Technologies, Shanghai Jiao Tong University, Northwestern University, Harbin Institute of Technology (Shenzhen), and the Shenzhen Loop Area Institute. The practical stakes are stated directly: interaction data are expensive to collect manually and difficult to verify and expand, which is why agentic data generation has become a central means of scaling both agent training and evaluation. Verifier accuracy is called out as especially critical for reinforcement learning, where the verifier directly shapes the reward signal and largely determines whether optimization reinforces genuinely successful behavior or exploits flaws in the reward design.

Future Directions

• Scaling under the ACE objective. The paper discusses scaling explicitly through the ACE lens, raising the question of how acceptance rates, learner-relative complexity calibration, and batch-level diversity trade off as generation volume grows.

• Data provenance. Provenance is named as one of the broader directions the paper discusses, relevant to whether environments were curated from real systems, synthesized by LLMs, or programmatically implemented — each carrying different realism, cost, and consistency properties.

• Earlier training stages. Agentic data are described as expanding beyond fixed post-training trajectories toward pre- and mid-training supervision, which raises open questions about what should be verified and calibrated when the learner is not yet a competent agent.

• Continual self-evolution beyond the task-level setting. The paper points to closed-loop experience that co-evolves with the learner and its environment, where accumulated experience, verification outcomes, model behavior, and observed coverage gaps continually revise generation strategies. This leaves open how to measure complexity relative to a moving learner and what to do where no complete verifier oracle exists and pipelines must fall back on partial rules combined with semantic review.

Target Audience

This paper is most useful to researchers and engineers building agent training data pipelines, agent evaluation suites, or tool and environment ecosystems, and to those who need to compare generation methods that are normally reported in incompatible, domain-specific terms. It is also valuable to practitioners deciding what to verify when constructing executable environments for reinforcement learning, since it separates construction mechanisms from the verification and selection decisions that determine whether generated experience is usable. Readers without a background in agent systems will find the organizing concepts accessible, but the formal notation and the taxonomy of named pipelines assume some familiarity with the current literature.

Authors’ abstract

LLM agents increasingly rely on generated interaction data to learn how to interact with external environments. Agentic data generation must maintain consistency among environments, tasks, interactions, and success signals while producing experience that is useful rather than merely abundant. Existing work spans many agent domains, but domain-centered organization and heterogeneous evaluation often obscure common generation mechanisms and conflate candidate construction with verification and selection. This work develops a two-level framework for the field. First, we represent agentic data as a common factorized object $(E,q,τ,v)$, comprising an environment specification, task signal, interaction realization, and optional verifier. We organize generation paradigms by their primary anchor and dependency structure. Second, we formulate generation as constrained distribution design through the Accuracy-Complexity-divErsity (ACE) lens. Accuracy establishes the feasible support of grounded and internally consistent data. Within this support, Complexity places learning mass relative to the capability of a declared learner and execution configuration, while divErsity controls coverage and redundancy of data. Using this framework, we explore how prior work verifies generated experience, constructs and calibrates difficulty, and expands behavioral coverage. The literature reveals a shift toward execution-grounded accuracy, learner-relative complexity, and diversity beyond surface variation or dataset size. We further discuss broader directions and emerging trends in agentic data generation through the ACE lens, including their implications for scaling, data sources, training regimes and adaptive learning. Overall, the central challenge is not simply to generate more data, but to continually allocate valid, informative, and non-redundant experience as agents and environments evolve.