Using Grounded Theory for Agent Behavior Analysis at Scale

AI summary

Overview

• Research area: Natural Language Processing / LLM agent behavior analysis, combining multi-agent LLM pipelines with qualitative research methodology (grounded theory).
• Technical level: Intermediate. The central idea is conceptual rather than mathematically heavy, but readers benefit from familiarity with LLM agent frameworks, trajectory-level datasets, and qualitative coding terminology.
• Scope: The paper introduces AutoTraceGT, a four-agent pipeline that automates grounded theory (open, axial, and theoretical coding) over LLM agent trajectories at scale, and evaluates it on 7,500+ trajectories across six environments and four backbone LLMs.

What This Paper Is About

As LLM agents tackle complex tasks (software engineering, web browsing, computer use, deep research), they still fail in ways that task-success metrics and simple metadata cannot explain. Existing analysis options are limited: numbers like trajectory length or action counts scale but say nothing about the problem-solving process, human qualitative analysis is interpretable but expensive, and pre-built behavioral classifiers are too rigid for novel tasks. The paper's goal is to bring grounded theory — a six-decade-old inductive qualitative method from the social sciences — into ML as a scalable, auditable way to describe what agents actually do.

Key Contributions

1. A new analytic method for ML. The authors argue that grounded theory, with its inductive stance and its principled saturation criterion, can serve as a productive analytic tool for scalable agent behavior and trajectory analysis.
2. AutoTraceGT. The authors build what they describe as the first end-to-end pipeline for automated grounded theory, implementing open, axial, and theoretical coding as role-specialized agents coordinated by a codebook manager, with every intermediate artifact machine-readable and auditable.
3. Empirical validation at scale. On 7,500+ trajectories across six datasets and four backbone LLMs, AutoTraceGT reaches saturation, produces reproducible codebooks, covers the majority of human-taxonomy failure modes, recovers prior expert theoretical accounts, and yields a deductive feature space for failure prediction.
4. A diagnostic account of what it adds beyond human taxonomies. The paper identifies three systematic reasons why its codebooks surface modes that pre-specified human taxonomies miss: granularity, cross-module structure, and catch-all labeling.

Main Findings

• Saturation is observed empirically. Across runs, add actions taper off while merge and confirm actions take over; category count plateaus at a dataset-specific value and cosine similarity to the terminal codebook approaches 1. This behavioral signature is also the algorithmic stopping signal.
• Codebooks are reproducible within a configuration. For each dataset–model configuration, AutoTraceGT was run three times on disjoint trajectory subsets. Within-configuration codebooks are significantly more similar than cross-configuration codebooks (Mann–Whitney U, p < 10⁻²⁰), and all configuration-level means exceed a data-derived reproducibility threshold estimated from cross-configuration comparisons.
• Structure is stable across backend models. Holding the dataset fixed, cross-LLM codebook coverage remains above null baselines that break the dataset correspondence, with permutation tests significant for every dataset (p < 0.001).
• Codebooks cover most human-annotated failure modes. On ALFWorld, recall is 75.0 and precision 60.0 against 15 human categories, with 20 AutoTraceGT categories, and 82.4% of individual trajectories' human-written reasonings matched. On GAIA, recall is 73.7, precision 63.2, with 11 human categories, 17 AutoTraceGT categories, and 58.0% trajectory reasoning match. On WebShop, recall is 90.9, precision 88.9, with 10 human categories, 13 AutoTraceGT categories, and 87.9% trajectory reasoning match. The paper states coverage is at least 70% on all three datasets, peaking at 90.9% on WebShop, and matches 58–88% of individual trajectories' reasonings.
• Recall can exceed precision, indicating discovered categories not in the human taxonomy. Three examples are given: ALFWorld's "noncompliant inaction" (distinguishable from invalid_action and impossible_action only by the complete absence of action); GAIA's "signaled-but-unrealized shifts" (a planning–execution decoupling that no single cognitive-module label can express); and WebShop's generic inefficient_plan label collapsing mode oscillation without synthesis into one bucket.
• Theoretical narratives converge with prior expert accounts. Without access to prior analysis, AutoTraceGT's theoretical coding recovers the same cascade-of-errors mechanism at the behavioral surface, with core categories of "feedback-decoupled control" (ALFWorld), "persistent repetition without adaptation" (GAIA), and "ritualized non-diagnostic search" (WebShop).
• Interpretive theoretical findings per benchmark. In SWE-Agent (T1), successful trajectories ground a proposed fix in executable evidence before modifying code, while failed trajectories use broad rewrites, unchecked assumptions, or superficial tests. In Go-Browse (T2), clicking visible UI affordances is not inherently predictive; what matters is whether a substantive verification step is inserted between action and closure. In Tau-Bench (T3), successful trajectories resolve task state before irreversible actions, while failed ones postpone alignment until after committing.
• GLM coefficients support the derived theories. For Go-Browse, clicking visible UI affordance is associated with success (β = −2.59), while clicking and closing without verification raises failure likelihood (β = +3.28). For SWE-Agent, "builds minimal reproducer after reconnaissance" is associated with success (β = −1.82) and "uses sweeping workaround" with failure (β = +4.10). For Tau-Bench, "executes after consent and reconciliation" is associated with success (β = −0.99) and "finds payment error late; escalates" with failure (β = +1.64).
• Codebooks support downstream failure prediction. Using FLAML AutoML over presence and co-occurrence features, the AutoTraceGT codebook provides useful predictive signal but is not always a standalone replacement for baselines; combining it with a few-shot codebook performs best on some datasets, which the authors read as complementary high-level and low-level behavioral signals. Notable cases: on SWE-Agent with GPT-4.1-mini, AutoTraceGT codebook feature engineering reaches MCC 0.314 and ROC AUC 0.765, exceeding both baselines; on Go-Browse with GPT-5, complementary feature engineering reaches MCC 0.499 and ROC AUC 0.828; on Tau-Bench with GPT-5-mini, complementary feature engineering reaches MCC 0.306 and ROC AUC 0.699.
• Aggregate coverage figure differs between sections. The abstract reports that codebooks recover 73–91% of failure modes in human-annotated taxonomies, while the conclusion states 74–91%.

Methodology in Plain English

The authors take a well-established social-science workflow — read the data, label concrete incidents, group labels into categories, refine categories against new data, stop when new data stops producing new structure — and assign each stage to a different LLM agent.

• OpenCode works on one trajectory at a time, labeling behavioral incidents with records of the form (code, span, quote): a 2–5 word conceptual code, a sub-step span, and a short verbatim quote as evidence. It is instructed to abstract behavioral patterns rather than restate tool calls. Because trajectories can have hundreds of steps, messages are chunked and a segment memo carries analytic continuity across calls.
• AxialCode works on a batch of already-coded trajectories, grouping conceptual codes into categories and typed relations, and emitting an axial memo. Each category carries a textual definition, its supporting code records, and a status tag of success, failure, or both derived from the terminal status of its source trajectories.
• Manage implements constant comparison across rounds. The codebook is versioned as a tuple of categories, relations, and a revision log. For each new category it chooses add, merge, split, or flag; for each new relation it chooses confirm, extend, add, or contradict. The policy is merge-first: new categories align to existing structure whenever a compatible match exists. Every action is appended to the revision log.
• TheoreticalCode runs only after saturation, taking the stabilized codebook and revision log and producing a core category plus a narrative that arranges the remaining categories as conditions, contexts, strategies, and consequences. It introduces no new local evidence.

The stopping rule is operationalized as the number of newly added categories in a round, and the pipeline terminates when the additions fall below a threshold for consecutive rounds. Sampling proceeds round by round through the trajectory pool under a sampling policy, so that later batches are drawn conditioned on the running codebook.

Evaluation used two corpus types. The outcome-labeled corpora span Tau-Bench (customer-service tool use), Go-Browse (web browsing), and SWE-Agent (software engineering), with 2,000 sampled trajectories each carrying only a success/failure label. The failure-annotated corpora span ALFWorld (100 trajectories), GAIA (50), and WebShop (the count is cut off in the provided text), each paired with expert annotations of failure types and causal reasoning. For the coverage comparison, AutoTraceGT was run with GPT-5-mini on the same trajectories, and GPT-5 was used as an LLM judge to decide whether a candidate category covers the human-written reasoning for each trajectory. For prediction, each trajectory prefix was represented as a fixed-length binary vector of category-presence and relation co-occurrence features and fed to FLAML AutoML.

Why This Matters

The paper addresses a methodological gap rather than a benchmark score: the ML community has scalable metrics that explain little and interpretable human analysis that does not scale. Grounded theory supplies an inductive, auditable, generalizable alternative with a principled stopping criterion, and AutoTraceGT shows the process can be automated without discarding the trail from raw trajectory to theoretical claim.

Potential real-world applications described or directly implied by the paper:

• Agent debugging and development. Agent developers can obtain behavior-level explanations of why trajectories fail, such as "keeps acting without diagnostic checks," decomposed into sub-behaviors like ignoring its own clues and retrying without re-checking, with step-level quoted evidence.
• Evaluation design. The codebooks can inform behavior-aware evaluation metrics and richer, more human-centered descriptions of agent performance than task success alone.
• Training data construction. The paper suggests the derived patterns could help build targeted training data and identify behaviors that should be reinforced or discouraged.
• Process-level reward signals. The authors list process-level reward signals as a downstream use of AutoTraceGT artifacts.

Industry relevance: the pipeline is positioned as an offline corpus-analysis tool for teams studying agent behavior across long, unfamiliar tasks, and the authors explicitly note it issues many LLM calls per trajectory and runs to saturation rather than a fixed budget, making it unsuitable for per-trajectory online use. The authors also caution that codebook categories should be treated as hypotheses for human review rather than ground truth, and that applying the framework to logs containing user content would require separate consent and privacy review.

Future Directions

• Richer saturation criteria. The authors propose criteria that account for both the density of examples within each category and the stability of relationships between categories, so the algorithmic stopping rule more closely matches the methodological idea of saturation.
• Downstream training and evaluation support. Using AutoTraceGT to construct targeted training data, identify behaviors to reinforce or discourage, design behavior-aware evaluation metrics, and build process-level reward signals.
• Transfer beyond the current instantiation. The paper leaves to future work the transfer to other qualitative methodologies, to multi-agent or dialogue surfaces, and to non-English trajectories.
• Additional open questions raised by the limitations. All coding stages are LLM-performed, so codebooks can inherit backbone blind spots; cross-LLM stability does not rule out biases shared across frontier models. Coverage and prediction numbers are conditional on an LLM judge. The authors also state that the algorithmic stopping criterion operationalizes theoretical saturation but is not identical to the methodological notion.

Target Audience

This paper is most useful to ML researchers and agent developers who need to understand agent behavior beyond success rates, particularly those working with long, multi-step trajectories on unfamiliar tasks. It also speaks to researchers in human–computer interaction and computational social science interested in scaling qualitative methods with LLMs, and to practitioners evaluating agents who want auditable, evidence-linked behavioral categories rather than opaque scores. Readers seeking a new benchmark or a directly deployable online monitoring system will find the work is deliberately positioned as offline analysis, with the authors noting the computational cost and the interpretive caveats.

Authors’ abstract

Understanding agent behavior requires methods that scale to thousands of trajectories and surface new patterns in long, often unfamiliar tasks where pre-built classifiers fall short. We propose to bring grounded theory into agent trajectory analysis: a six-decade-old qualitative method from the social sciences, with a principled saturation criterion and an auditable trail from data to theory. We propose AutoTraceGT (Automated Trace analysis through Grounded Theory), the first multi-agent pipeline that automates grounded theory on agent trajectories. It iteratively performs open, axial, and theoretical coding until saturation, producing a behavioral taxonomy tailored to each task. Across six trajectory corpora, AutoTraceGT produces codebooks that recover 73-91 percent of the failure modes in human-annotated taxonomies and surface additional patterns that those taxonomies miss. The emergent theoretical narrative aligns with prior expert accounts. Used as a deductive feature space, the codebook outperforms zero-shot and few-shot LLM baselines on downstream failure prediction. These results suggest Grounded Theory offers a scalable analytic tool for ML researchers and agent developers studying what agents actually do.