Circuit Oracle: Automating Attribution Graph Analysis via Natural-Language Queries

AI summary

Overview

Research area: Mechanistic interpretability / AI safety tooling for large language models, specifically the automated analysis of attribution graphs.

Technical level: Advanced. The paper assumes familiarity with transcoders, sparse autoencoders, attribution graphs, causal interventions, and linear probes.

Scope: The paper introduces Circuit Oracle, an agentic system built on an off-the-shelf frontier LLM that answers natural-language questions about a model's attribution graph through tool calls, and evaluates it on three safety-relevant proxy tasks: spurious-feature detection, secret elicitation, and suppression jailbreaking.

What This Paper Is About

Attribution graphs decompose a language model's computation into sparse interpretable features joined by causal edges, but turning such a graph into a safety-relevant insight currently takes experts hours of manual reading. The authors ask whether an off-the-shelf LLM, given graph-traversal tools and a written task specification ("skill"), can answer high-level natural-language questions about an attribution graph without any additional training. They test this on three proxy tasks and compare against activation-only baselines and against trained activation readers.

Key Contributions

1. A circuit-oracle framework. The authors propose "circuit oracles" — analogous to activation oracles — that answer natural-language questions about attribution graphs. The implementation is an off-the-shelf frontier LLM with a multi-agent harness (an orchestrator dispatching subagents with hop budgets) plus task-specific "skills," each skill being a static system prompt bundled with shared graph-traversal tools and task-specific tools, including interventions that steer transcoder features and re-run the target model.

2. Three safety-relevant proxy-task evaluations. Spurious-feature detection on probe circuits (Gemma-2-2B with Gemma Scope PLTs), secret knowledge elicitation from taboo-finetuned models (Qwen-3-8B with LoRA adapters from Karvonen et al., 2025), and suppression jailbreaking via feature-level intervention (Qwen3-4B on 50 refusal prompts).

3. Analysis artifacts beyond task scores. For every run the oracle emits a pruned subgraph and context-specific feature explanations, which the authors argue revise existing global auto-interp labels; they release these subgraphs and readings and invite evaluations that measure their usefulness directly.

4. Ablations and stability analyses. A tool ablation removing edge traversal, a sweep over three orchestrator models, five repeat passes per graph, and per-prompt win/tie/loss breakdowns against baselines.

Main Findings

• Spurious-feature detection: modest win. Circuit Oracle reaches 78.2% ± 2.4 accuracy (5-pass mean), against Trans-cos at 72.5% and SAE-cos at 65.0%. Individual passes range from 76.2% to 82.5%. Both baselines are near-perfect on biased rankings (40/40 each) and weak on unbiased ones, scoring 0/10 unbiased on CivilComments and MultiNLI.

• Traversal is not load-bearing on the probe task. A one-shot readout with no edge traversal reaches 77.5%, statistically indistinguishable from the full agentic pipeline's 78.2% ± 2.4. The authors attribute this to the spurious cue sitting close to the output.

• Traversal is load-bearing on jailbreaking. On suppression jailbreaking the same one-shot ablation falls far short of the full pipeline's 0.804 (Appendix G.4).

• Secret elicitation: beats projection baselines, trails trained readers. Circuit Oracle reaches 75.0% overall closed-mode accuracy, ahead of SAE-proj (25.8%) and Trans-proj (1.7%). However, the Activation Oracle baseline reaches 86.7% closed and 74.2% open over the same 20 secrets. The introduction separately describes the gap as 69.2% vs. 81.2% closed-menu; the paper reports both framings without reconciling them.

• Suppression jailbreaking: roughly on par, slightly behind on a single pick. Against the diff-in-mean refusal-direction baseline of Arditi et al. (2024) on 50 prompts: no intervention 0.041, Arditi 0.825, Circuit Oracle grader's pick 0.804 (10 wins / 20 ties / 20 losses), and best-of-five 0.854 (16 wins / 24 ties / 10 losses). The authors call the single-pick comparison the fairer one.

• Verdicts are stable, circuits less so. Across five passes over fixed graphs, two passes agree on which features they pin only at Jaccard 0.511, though they agree much more on which layers carry the evidence (Jaccard 0.754). Swapping the orchestrator moves accuracy by an amount comparable to five passes of the same model.

• Global labels can mislead. Appendix G.7 documents a refusal-precursor feature whose Neuronpedia auto-interp label disagrees with the causal role recovered through targeted intervention and inspection of top-activating prompts.

Methodology in Plain English

The starting point is an attribution graph: a directed graph whose nodes are sparse, human-interpretable "transcoder features" (plus token embeddings and output logits) and whose edges carry quantitative direct-effect weights computed via Jacobians under a linearized "local replacement model." Such a graph for a short prompt can have hundreds of active features across dozens of layers and thousands of nonzero edges.

Rather than training a model to read activations, the authors hand an off-the-shelf frontier LLM a set of tools for walking this graph — retrieving the strongest upstream nodes, inspecting features via Neuronpedia, following causal edges — and pair it with a written "skill" for the task at hand (a system prompt plus task-specific tools). The implementation uses a multi-agent setup: an orchestrator (Minimax-M3) that dispatches subagents (GPT-OSS-120B), each starting from a feature, tracing upstream hop by hop within a hop budget, and reporting discovered features, edges, and an explanation. Findings are consolidated into a pruned circuit plus a natural-language analysis.

For spurious-feature detection, they train biased probes (trained only on stereotype-consistent examples) and unbiased probes (trained on balanced splits) on mean-pooled residual streams at a chosen layer, on BiasInBios, CivilComments, and MultiNLI, producing 4 tasks × 2 probes × 10 prompts = 80 attribution graphs. An LLM judge (gpt-5.4-mini) counts named spurious- versus core-aligned features and returns a categorical verdict.

For secret elicitation, they extract a transcoder attribution graph from each taboo-finetuned model and a sibling graph from the base model on the same prompt, then use a cross-graph diff to cancel template scaffold features and a vote tally to match surviving features' top logits against a 20-word closed menu.

For jailbreaking, the orchestrator nominates features for multiplicative steering at scales −1 through −4 under a monotonicity rule, verifies each by re-running the subject model, and commits to one intervention. Completions are scored by a five-model judge ensemble (claude-sonnet-5, grok-4.3, gemini-3.5-flash, kimi-k2.6, glm-5.2) on usability and plausibility in [0,1], averaged.

Why This Matters

The paper tests whether a general-purpose, training-free agent can substitute for task-specific interpretability methods, and finds the answer is "sometimes." That is a useful negative-and-positive result for a field where most automated interpretability systems bind the analyzer to a specific subject architecture via training.

Real-world applications:

• Red-teaming and auditing: an auditor could point the oracle at a circuit and ask "is this prediction driven by spurious features?" without hand-writing analysis.
• Fairness debugging of probes: identifying when a linear classifier reading a model's internals has latched onto gender markers, identity-group mentions, or negation cues rather than task content.
• Model-diffing after fine-tuning: using base-model sibling graphs to surface features a fine-tune drives harder than the base, as in the taboo setup.
• Extending to new safety questions cheaply: writing a new skill rather than fine-tuning a dedicated reader.

Industry relevance: The skills-and-tools pattern is a low-friction way for model developers to add interpretability triage without a bespoke training pipeline. The finding that auto-interp labels can misrepresent a feature's causal role argues for in-context, task-conditioned reading over static global labels in production tooling.

Future Directions

• Evaluations for the artifacts, not just the verdicts. The authors explicitly invite work measuring the usefulness of the released pruned subgraphs and context-specific feature readings, since current proxy-task scores do not capture what those artifacts buy.
• Whether differently-featured circuits behind the same verdict are semantically equivalent. The Jaccard 0.511 feature-level agreement across passes is not treated as instability by the authors, but they state this task cannot settle the question.
• Closing the gap to trained activation readers. Activation Oracles exceed the graph-reading oracle on both closed (86.7%) and open (74.2%) secret elicitation, suggesting privileged in-distribution access to activation geometry still matters.
• Better commit rules for intervention. The gap between the grader's single pick (0.804) and best-of-five (0.854) bounds what a better selection rule could recover.
• Diagnosing the MultiNLI failure mode, where the oracle missed negation cues because content words outnumbered the sparse negation signal in its reports.

Target Audience

Interpretability researchers and mechanistic-interpretability engineers who already understand transcoders, SAEs, and causal interventions and want to know whether agentic LLM analysis is a viable alternative to trained readers. Also relevant to AI safety auditors and red-teamers interested in reusable, skill-based tooling, and to evaluation researchers looking for proxy tasks that stress interpretability methods rather than circuit-quality metrics. Beginners will find the appendix background on attribution graphs useful but the main text assumes substantial prior context.

Authors’ abstract

Attribution graphs, an emerging tool in mechanistic interpretability, use transcoders to decompose language model computations into sparse interpretable features connected by causal edges. However, turning a graph into a safety-relevant insight requires hours of manual analysis by experts. We introduce Circuit Oracle, an off-the-shelf LLM equipped with a multi-agent harness and task-specific authored skills, which answers natural-language questions about a target model by reading the attribution graph through tool calls. We evaluate it on three safety-relevant proxy tasks: detecting spurious features in probe circuits, eliciting hidden knowledge from taboo-finetuned models, and suppression jailbreaking via feature-level interventions. Beyond task scores, the oracle produces pruned subgraphs and context-specific feature explanations that we assess qualitatively. Results are mixed. On spurious-feature detection, Circuit Oracle reaches 78.2 $\pm$ 2.4% accuracy, outperforming single-layer feature-ranking baselines. On secret elicitation, trained self-explainers like Activation Oracles remain the stronger baseline. On refusal suppression over 50 prompts, its single committed intervention is slightly below a diff-in-means baseline, and only exceeds it when allowed the best of five attempts. Results are stable across five runs and across model choices for the agents. We also release the pruned subgraphs and context-specific feature readings that revise existing global labels, and invite future work to develop evaluations that can measure their usefulness directly.