AgentXRay: White-Boxing Agentic Systems via Workflow Reconstruction

AI summary

Overview

Research area: Interpretability and reverse-engineering of LLM-based agentic systems — specifically the reconstruction of explicit workflows from black-box input–output behavior.

Technical level: Advanced. The paper assumes familiarity with Monte Carlo Tree Search, combinatorial optimization, LLM agent frameworks, and AST-based code similarity metrics.

Scope: The paper defines a new task, Agentic Workflow Reconstruction (AWR), and proposes AgentXRay, an MCTS-based search framework with a "Red-Black" pruning mechanism that synthesizes an editable, interpretable stand-in workflow approximating a black-box agent system using only input–output access.

What This Paper Is About

Many deployed LLM agentic systems are opaque black boxes: users can query them but cannot see the agent roles, prompts, tool calls, or coordination topology inside. This paper asks whether an explicit, white-box workflow can be recovered purely from a dataset of input–output pairs collected from such a system. The goal is a surrogate workflow that is behaviorally similar (by an output-based proxy metric) and editable, rather than a parametric copy like a distilled model.

Key Contributions

1. A new task — Agentic Workflow Reconstruction (AWR). The paper formalizes converting black-box LLM agent systems into white-box counterparts by reconstructing explicit workflows from input–output observations, applicable to both multi-agent and complex single-agent systems.

2. A method — AgentXRay. An MCTS-based search framework over a Unified Primitive Space of agentic primitives, equipped with a Red-Black Pruning mechanism that dynamically colors nodes based on Quality, Depth, and Width scores to decide between depth refinement and width expansion.

3. An evaluation across five domains. AgentXRay is tested on code generation (ChatDev), data analysis (MetaGPT), education (TeachMaster), scientific computing (Gemini 3 Pro), and 3D modeling (ChatGPT / GPT-5.2, both via public APIs), using 52 SRDD tasks, 52 MatPlotBench tasks, 25 teaching video tasks, 80 SciBench problems, and 100 ScanRefer tasks respectively.

4. A search-space complexity analysis plus an open-weight generalization study. The paper derives contraction bounds for the pruned search space and separately reconstructs Atoms (a multi-agent platform evolved from MetaGPT) using 80 presentation generation tasks from the Automatic Slide Generation dataset, with a primitive space built exclusively from open-weight models.

Main Findings

• Best average reconstruction fidelity: AgentXRay reaches an average Static Functional Equivalence (SFE) of 0.426, versus 0.339 for the unpruned MCTS baseline AFlow and 0.299 for Claude Opus 4.5 + ReAct, under the same interaction budget.

• Pruning is decisive, not incidental: Removing Red-Black Pruning drops the average to 0.318 and can underperform AFlow (e.g., ChatDev: 0.286 vs. AFlow's 0.403). The paper attributes this to the expanded branching factor of the unified primitive space dispersing a fixed budget across low-quality branches.

• Pruning enables deeper workflows: Without pruning, MCTS stagnates at workflow length L = 2 across all five domains. With pruning, search reaches maximum depth L = 6, with Red node proportions between 45% and 65%.

• Token savings of 8–22%: Red-Black Pruning reduces token consumption by 8% to 22% across all five domains versus unpruned search, with the largest reduction on ChatDev (22.3%, from 17.16M to 13.33M tokens).

• Domain variance: MetaGPT is the most reconstructable target (0.557 with All Tools), while ChatDev shows higher variance due to multi-file dependency sensitivity. MetaGPT is also the most token-efficient; the remaining three domains cluster around 2.5–3.0M tokens.

• Behavior cloning fails: SFT performs worst at an average of 0.196, especially on Education (0.124) and 3D Modeling (0.091), suggesting input→output cloning does not recover procedural structure.

• Model capacity is not a substitute for structure: A stronger single model (Claude Opus 4.5) reaches 0.292, and adding ReAct-style tool use raises this only to 0.299, even though the Claude baselines use a stronger model than any component in AgentXRay's primitive space.

• Tool integration matters unevenly: Removing tools lowers the average from 0.426 to 0.356 (−16.4%), with a large drop on MetaGPT (−46.0%) but a marginal one on Scientific Computing (−4.3%).

• Convergence speed: On ChatDev, the "Sel. Tools" variant reaches a 0.65+ score at roughly 10M tokens, whereas "w/o Pruning" needs roughly 12M tokens to reach the same level.

• SFE validated by humans: A blind study with three annotators on 30 stratified-randomly sampled output pairs found human similarity scores positively correlated with SFE (Spearman ρ = 0.61, p < 0.001, 95% CI [0.30, 0.80]), with moderate inter-annotator agreement (Krippendorff's α = 0.57).

• Open-weight models suffice: On Atoms, AgentXRay (All Tools) achieves the best score of 0.337, outperforming AFlow (0.321) by 5.0%, indicating results do not depend on privileged access to proprietary models.

• Statistical robustness: Across 5-seed runs with paired t-tests against AFlow, the relative ranking of methods is preserved on all seeds and even worst-case runs exceed AFlow.

Methodology in Plain English

The researchers start by assuming they can only see what goes into a black-box agent system and what comes out — no prompts, no roles, no tool lists. They then define a vocabulary of "agentic primitives," where each primitive bundles a role, a base model, a thought pattern, and an optional set of local tools. Both pure-reasoning agents and tool-augmented agents fit into this single vocabulary.

Rather than searching over arbitrary graph topologies (which the paper notes scales as O(2^(|Ω|²))), they adopt a "linearity hypothesis" and represent workflows as ordered chains of primitives, arguing that graph-structured systems still serialize into a concrete execution-time ordering. The reconstruction problem becomes: find the chain of primitives whose outputs best match the black-box outputs.

To search this space, they use Monte Carlo Tree Search, where each tree node is a partial workflow prefix and each edge appends one primitive. Because similarity can only be measured after a workflow is (nearly) fully executed, MCTS handles the delayed reward naturally. Each iteration does selection/expansion, a rollout that samples primitives to completion, and backpropagation of the reward.

The distinctive piece is Red-Black Pruning. Every node gets a composite score: average reward (Quality) multiplied by normalized depth (Depth) multiplied by the fraction of children created so far (Width). Nodes above a β-quantile threshold are colored Red and continue refining depth by choosing among existing children with UCB; nodes below are colored Black and instead expand by creating new children. Nodes at maximum depth or marked terminal from execution failure are excluded. This concentrates the limited iteration budget on promising prefixes instead of spreading it across many branches.

Evaluation uses Static Functional Equivalence (SFE), an AST-derived similarity metric combining interface similarity (20%; signatures and module structure), logic similarity (50%; control flow and algorithmic patterns), and semantic similarity (30%; identifier semantics and code-intent features), with a TF–IDF cosine fallback when AST parsing fails (<5% of samples).

Why This Matters

Impact on research. The paper opens a complementary line to model distillation and to white-box workflow generation like AFlow: instead of transferring behavior into parameters or optimizing a known architecture, it recovers an explicit, editable workflow from behavior alone. It also connects in spirit to model extraction via prediction APIs, but delivers a workflow rather than a parametric replica. For interpretability research, it moves beyond post-hoc explanations of individual decisions toward reconstructing an executable artifact that can be inspected, edited, replayed, and debugged.

Real-world applications (bullets):

• Auditing and debugging vendor agents. Teams that call proprietary assistants through public APIs could reconstruct a stand-in workflow to understand and troubleshoot behavior they cannot otherwise see.
• Cost and dependency reduction. A reconstructed white-box workflow can serve as an editable, potentially cheaper substitute for an expensive or rate-limited external system.
• Safety-critical deployment review. In domains where decision opacity blocks deployment, an explicit surrogate workflow provides something reviewers can inspect and modify.
• Cross-domain reuse of proven pipelines. Workflows recovered from strong systems in data analysis, scientific computing, education, or 3D scripting can be adapted rather than rebuilt from scratch.

Industry relevance. The ability to duplicate functionality from black-box query access raises the same dual-use questions as prediction-API model extraction, which the paper explicitly acknowledges. Practically, it gives organizations a path to regain control over agentic systems they depend on but do not own, and a tangible artifact for compliance and governance discussions.

Future Directions

• Graph-structured workflow representations. The chain abstraction cannot faithfully capture true concurrency or asynchronous coordination, such as parallel tool calls, event-driven updates, or branching with multiple simultaneously active threads. The paper points to richer representations like typed DAGs.
• Denser feedback and earlier credit assignment. Step-level reward models, outcome verification models, or learned value functions could reduce reliance on expensive full rollouts and improve search efficiency beyond the observed 8–22% token reduction.
• Extension to purely natural language domains. Creative writing and open-ended dialogue would require new evaluation approaches, combining embedding-based semantic similarity with human judgment or LLM-as-judge protocols.
• Theoretical limits of reconstructibility. The paper states that establishing bounds on what can be reconstructed remains open, particularly when targets rely on proprietary tools or hidden capabilities unavailable to the reconstruction space.

Target Audience

Researchers working on LLM agent interpretability, multi-agent system design, and automated workflow search will get the most from this paper, as will practitioners who depend on black-box agent APIs and need to understand, audit, or replace them. It is also relevant to readers interested in the intersection of MCTS-based search and LLM systems, and to those thinking about the dual-use implications of extracting behavior through prediction APIs. The paper is not beginner-friendly: it assumes comfort with MCTS, UCB, combinatorial search, and code-similarity metrics.

Authors’ abstract

Large Language Models have shown strong capabilities in complex problem solving, yet many agentic systems remain difficult to interpret and control due to opaque internal workflows. While some frameworks offer explicit architectures for collaboration, many deployed agentic systems operate as black boxes to users. We address this by introducing Agentic Workflow Reconstruction (AWR), a new task aiming to synthesize an explicit, interpretable stand-in workflow that approximates a black-box system using only input-output access. We propose AgentXRay, a search-based framework that formulates AWR as a combinatorial optimization problem over discrete agent roles and tool invocations in a chain-structured workflow space. Unlike model distillation, AgentXRay produces editable white-box workflows that match target outputs under an observable, output-based proxy metric, without accessing model parameters. To navigate the vast search space, AgentXRay employs Monte Carlo Tree Search enhanced by a scoring-based Red-Black Pruning mechanism, which dynamically integrates proxy quality with search depth. Experiments across diverse domains demonstrate that AgentXRay achieves higher proxy similarity and reduces token consumption compared to unpruned search, enabling deeper workflow exploration under fixed iteration budgets.