White Box Evidence Packages for Policy Audit Reports

AI summary

Overview

Research area: Technical AI governance and mechanistic interpretability, specifically how evidence from a model's internals ("white box" evidence) is used inside LLM-generated policy audit reports.

Technical level: Intermediate. The paper is written for readers comfortable with AI auditing terminology and interpretability tools (sparse autoencoders, logit lens, activation patching), though the audit framing is explained in plain terms.

Scope: A controlled study holding the policy passage, rubric, and auditor model fixed while varying only the evidence interface supplied to a local Qwen 2.5 7B auditor, across 60 AGORA policy cases and 600 generated reports under ten evidence conditions.

What This Paper Is About

As AI governance moves from benchmark scores toward auditable oversight, reviewers need to tell whether an LLM-written audit report is actually supported by the evidence it cites. This paper studies "passage anchored policy audits," where a report must interpret a given policy passage and cite evidence for its claims, and asks what changes when the evidence supplied to the auditor includes internal model evidence about a target model. The goal is to evaluate internal model access as an evidence design problem for audit workflows, rather than as a guarantee of transparency.

Key Contributions

1. A controlled evaluation framework for evidence interfaces. The passage, rubric, and auditor model are held fixed while only the evidence interface changes, across 60 AGORA policy cases and 600 structured reports under ten conditions — including passage-only, black box surface evidence, sparse autoencoder evidence, logit lens evidence, steering sensitivity evidence, an activation explanation surrogate, combined white box evidence, hybrid surface-plus-white-box evidence, raw AutoInterp sparse autoencoder evidence, and a shuffled white box relevance control.

2. A validation layer over report quality. Five reviewers contribute across gold writing, verification, diagnostic scoring, and agreement checks, producing 240 scored reports (all 60 cases under four primary interfaces) on correctness, span grounding, diagnostic usefulness, and evidence misuse.

3. A separate behavior-locked residual stream patching diagnostic on Gemma 2 2B that tests whether candidate causal localization is stable across prompt shells and answer orders, rather than treating report citations as causal evidence.

4. An operationalized negative control. A shuffled white box relevance control preserves the evidence format while pairing internal evidence with the wrong case, exposing a governance risk: reports can sound substantively plausible while citing irrelevant internal evidence. Code, prompts, evidence packages, final reports, validation review files, and figures are released at https://github.com/GRAIL-center/agora-mi.

Main Findings

• Internal evidence changes citation behavior, but citation volume is not validity. The combined white box interface cites 10.13 evidence entries per report versus 3.87 under black box surface evidence, yet the shuffled relevance control also yields 9.65 citations per report.

• Combined white box evidence weakens grounded quality. It preserves high average correctness (4.60 vs. 4.68 for the surface baseline), but weakens grounding (3.25 vs. 4.52), lowers usefulness (4.00 vs. 4.68), and increases misuse (2.50 vs. 1.00). In paired counts it is worse than the surface baseline on grounding in 53 of 60 cases and worse on usefulness in 41 of 60, with misuse higher in all 60 cases.

• The hybrid interface is the strongest design signal, but it is confounded with package size. Hybrid is nearly tied with the surface baseline on correctness (4.70 vs. 4.68) and grounding (4.50 vs. 4.52), while improving usefulness from 4.68 to 4.92 (positive in 17 cases, tied in 40, negative in 3). It also increases misuse in all 60 cases, from 1.00 to 2.13. The hybrid package averages 22.97 evidence items, compared with 3.27 for black box surface evidence and 19.70 for combined white box evidence.

• The auditor reads white box evidence shallowly rather than ignoring it. In raw citations, the combined interface cites logit lens most often (232 citations vs. 68 for sparse autoencoder). Normalized by availability, sparse autoencoder labels dominate: 93.2% of available SAE entries cited, versus 33.9% for logit lens, 3.1% for steering, and 2.8% for activation explanation surrogate entries. The shuffled control cites 95.9% of available SAE entries despite the evidence coming from another case. In the optional 20-case single-tool review, logit lens evidence and raw AutoInterp sparse autoencoder evidence each have mean grounding 1.25 and evidence misuse 4.00.

• The causal localization is narrow and rendering sensitive. In the direct shell, canonical order run, deontic force passes the causal gate at the final prompt token at layer 7 (mean recovery 0.1968, 95% bootstrap interval [0.0302, 0.3968]) and layer 8 (0.1429, [0.0220, 0.2819]), against an empirical threshold of 0.0935. Authority assignment has no passing region. The direct shell, swapped order control also passes at layer 10, while the analyst shell, canonical order control has no passing region. The paper reads this as a prompt-sensitive late decision or answer readout signal.

• The shuffled control is the strongest warning. Thirty-nine of sixty shuffled control reports have correctness at least 4 while also receiving evidence misuse at least 4. Only 3 of those 39 reports warn about weak evidence, and only one explicitly names an internal tool. Shuffled condition means are correctness 3.63, grounding 2.67, usefulness 2.63, misuse 5.00, with misuse higher than the surface baseline in all 60 cases (mean delta +4.00).

• Package size correlates with misuse in the validation records. Package size is positively associated with evidence misuse (Spearman rho = 0.455) but not with diagnostic usefulness (rho = -0.029); the authors describe this as not causal evidence.

• Qualitative cases show the same split. On Articles 29 and 30 of the Chinese Algorithmic Recommendation Provisions, the hybrid interface covers both confidentiality and complaint handling (5, 5, 5, 2) while the shuffled control is partly correct but cites shuffled transparency evidence (4, 2, 3, 5). On the EU AI Act Annex III classification exception, hybrid improves anchoring but can still use imprecise low risk language (4, 4, 4, 2), which is why the paper does not claim hybrid evidence solves legal precision.

Methodology in Plain English

The researchers built a pipeline where each of 60 AGORA policy cases (drawn from 11 source documents and covering obligations, rights, exceptions, enforcement mechanisms, and governance procedures) is paired with a fixed audit rubric and one evidence package. A local Qwen 2.5 7B auditor writes a structured report from the passage, rubric, and package only; a hidden gold brief is withheld and used only for review.

The evidence packages describe a target model, Gemma 2 2B. Internal evidence is generated deterministically with fixed tools: Gemma Scope residual stream SAEs with human revised AutoInterp labels, logit lens summaries at layers 6, 12, 18, and 24, steering directions at layers 12, 18, and 24 using a fixed obligation and risk contrast prompt set, and a conservative activation explanation surrogate. Ten conditions vary only the interface shown to the auditor, including a shuffled white box relevance control that keeps the combined package format but draws internal evidence from another case, with the auditor not told.

Reports are normalized into findings, passage spans, cited evidence, internal evidence use, confidence, and cautionary notes, with structural checks that citations were actually supplied. Five reviewers then score reports on a 1-to-5 scale for correctness, span grounding, usefulness, and evidence misuse (lower is better only for misuse), covering all 60 cases under four primary interfaces (240 reports), with agreement diagnostics on the original 20-case subset and single-tool diagnostics on that subset. Paired comparisons against the black box surface baseline use exact sign tests and Wilcoxon signed rank approximations as descriptive checks.

Separately, a behavior-locked residual stream patching diagnostic uses 60 forced-choice microtasks (30 authority assignment, 30 deontic force), split into 18 train, 6 validation, and 6 test pairs and rendered with direct and analyst prompt shells and canonical and swapped answer orders. Both families pass a pre-patching behavior gate of rendered accuracy at least 0.70 and at least 20 stable pairs out of 30 (authority 0.958 with 29 stable pairs; deontic 0.796 with 22). Clean residual states are patched into corrupted prompts across Gemma 2 2B's 26 layers and five token roles, with 64 random layer and token role trials per pair; a region passes only if mean recovery exceeds the family random region 95th percentile by more than 0.05 and the lower bound of a 1,000-resample 95% bootstrap interval exceeds the random region mean.

Why This Matters

Impact on research. The paper reframes internal model access as an evidence design problem rather than a transparency guarantee, and supplies a reproducible interface-level framework plus negative control. It also argues that readable interpretability outputs should be validated through downstream report use, since report citations are a shallow semantic signal rather than mechanistic support.

Real-world applications.

• Regulatory and assurance audits (the paper cites regulatory audits, AI safety institute evaluations, DSA-style platform audits, and assurance standards) where conclusions must trace to sufficient and appropriate evidence.
• Designing reviewer-facing evidence packages for LLM-assisted audit workflows, where reviewers need to know which evidence was available, what was cited, and whether irrelevant evidence would have produced similar confidence.
• Building relevance controls into audit tooling so that plausible-sounding reports citing irrelevant internal evidence do not pass unchecked.
• Prioritizing which interpretability artifacts to expose, given that the auditor heavily cites readable sparse autoencoder labels and logit lens directions while steering and activation explanation entries are cited rarely.

Industry relevance. Organizations deploying internal-access evaluation tools for compliance, risk assessment, or model assurance can use the finding that citation volume is not a quality measure and that package size correlates with evidence misuse. The paper explicitly does not propose an automated legal compliance system and does not test whether human auditors would be fooled by shuffled evidence.

Future Directions

1. A capped hybrid condition to separate anchoring from package size, since the current hybrid advantage is confounded with evidence quantity and variety.
2. A hybrid plus shuffled condition to test whether surface anchors protect against irrelevant internal evidence.
3. Numeric-only or label-masked internal evidence to test whether readable labels drive over-trust, and a visible-but-not-citable condition to separate evidence influence from citation behavior.
4. Running another auditor model or a stricter prompt to test whether the pattern is Qwen-specific, alongside addressing known limitations: reviewers scored only cleaned reports, agreement diagnostics cover only the original 20 cases, single-tool interfaces are human scored only on that subset, the target model is small, the activation explanation tool is a surrogate rather than a compatible real Activation Oracle, and SAE label quality, logit lens noise, and steering direction quality are not calibrated.

Target Audience

Technical AI governance researchers and policy auditors, mechanistic interpretability researchers who want application-level evaluation of tool outputs, and evaluation or assurance teams building auditable LLM-assisted review pipelines. Readers should have some familiarity with audit evidence concepts or interpretability tools; the paper's control structure and reporting are explained without requiring deep mathematical background.

Authors’ abstract

As AI governance moves from benchmark scores toward auditable oversight, a central question is how reviewers can tell whether an LLM-generated audit report is actually supported by evidence. This paper studies that question in passage-anchored policy audits, where a report must interpret a given policy passage and cite evidence for its claims. We introduce a controlled evaluation framework that holds the passage, rubric, and auditor model fixed while changing only the evidence interface supplied to the auditor. Across 60 AGORA policy cases, we generate 600 structured reports under ten evidence conditions, including passage-based evidence, internal model evidence, a hybrid package, and a shuffled control that preserves evidence format while breaking case relevance. Five human reviewers evaluate the primary interfaces for correctness, passage grounding, diagnostic usefulness, and evidence misuse. The results show that internal evidence changes how reports cite and reason about evidence, but more internal citations do not by themselves make a report more valid. A white-box diagnostic explains the failure mode: causal localization is narrow, while reports readily reuse broader readable labels and token directions. The hybrid interface is the most useful on average, while the shuffled control exposes a key governance risk: reports can sound substantively plausible while citing irrelevant internal evidence. This study reframes internal model access as an evidence design problem for audit workflows, rather than as a guarantee of transparency.