Replayable Financial Agents: A Determinism-Faithfulness Assurance Harness for Tool-Using LLM Agents

AI summary

Overview

Research area: Reliability, reproducibility, and auditability evaluation for tool-using LLM agents deployed in regulated financial workflows.

Technical level: Advanced. The paper assumes familiarity with LLM agent architectures (ReAct-style tool calling), temperature-zero sampling, bootstrap confidence intervals, partial correlation, and benchmark evaluation conventions such as pass@k.

Scope: The paper introduces and empirically validates the Determinism-Faithfulness Assurance Harness (DFAH), a measurement framework that formalizes and measures whether financial LLM agents can reproduce decisions and tool trajectories under identical inputs.

What This Paper Is About

When a regulator asks why a transaction was flagged, a financial institution must show that the decision can be replayed with identical inputs and that it was grounded in retrieved evidence rather than fabricated reasoning. Most deployed agents fail this test: they produce different tool sequences or even different final decisions across repeated runs, even at temperature zero. This paper defines measurable properties for "replayability," builds an open-source harness to measure them, and tests whether an agent being deterministic tells you anything about whether it is accurate.

Key Contributions

1. The DFAH framework. Formal definitions of action determinism, signature determinism, decision determinism, evidence grounding, and constraint satisfaction for tool-using agents, plus an open-source implementation structured around the trial/transcript/grader terminology now standard in production agent evaluation.

2. A null correlation result. Across 21 model-benchmark configurations, decision determinism and task accuracy show no detectable relationship (Pearson r = −0.11, 95% BCa CI [−0.49, 0.31], p = 0.63; Spearman ρ = −0.08; partial correlation controlling for model size, task type, and inference environment r = −0.01). This is the paper's central argument: neither metric can proxy for the other, so both must be measured separately.

3. A large-scale agentic evaluation. 4,705 measured runs across 7 models, 4 providers, and 3 financial benchmarks of 50 cases each at T = 0.0, spanning compliance triage, portfolio constraint checking, and DataOps exception handling.

4. A determinism-accuracy tradeoff characterization and deployment guidance. A tiered model profile, validation scaling factors (up to 3.7× more validation samples for the least deterministic tier), stress-test protocols, and an argument that pass^k (all trials must succeed) rather than pass@k (at least one succeeds) is the compliance-relevant metric.

Main Findings

• Determinism is not accuracy. The hypothesis that a deterministic agent is also an accurate one is not supported. The paper reports three distinct operational profiles: high determinism with low accuracy (small local models), moderate determinism with variable accuracy (frontier models), and variable determinism with variable accuracy (Gemini 2.5 Pro). No model occupies the high-determinism, high-accuracy quadrant.

• Small models buy determinism through pattern matching, not understanding. The 7–20B Tier 1 models reach 94–100% decision determinism but only 20–42% accuracy. Qwen2.5:7b defaults to "investigate" on 76% of compliance alerts and "modify" on 82% of portfolio trades regardless of case content. The paper quantifies this as decision concentration: normalized entropy H/log|D| of 0.56 for small models versus 0.94–0.98 for frontier models such as Claude Opus 4.5 and Gemini 2.0 Flash.

• Frontier models reach the same conclusion by different routes. Across all three benchmarks, frontier models pair moderate-to-high decision determinism (71–86%) with much lower signature determinism (20–58%). Decision replay succeeds while trajectory replay fails, meaning evaluators must be explicit about which level of reproducibility their regulatory context requires. Claude Opus 4.5 achieved the highest compliance triage accuracy (68.7%) at 72% decision determinism but only 44% signature determinism.

• Decision replay and trajectory replay diverge by task structure. Decision determinism is comparable across the three financial benchmarks (79–84%), but signature determinism drops substantially for semi-structured work, falling to 56.8% on DataOps Exception. Tool-path variance, not decision variance, is the primary source of non-reproducibility in agentic settings.

• Errors can be deterministic too. On DataOps Exception, Claude Sonnet 4 posted 84% decision determinism at only 14% accuracy by defaulting to "quarantine" on 94% of cases. The authors note that for audit-critical deployments, consistent errors may be preferable to accurate but unpredictable variance.

• An earlier positive correlation was a measurement artifact. Version 1 of this work reported r = 0.45 between determinism and faithfulness using reconstructed data that conflated trajectory and decision determinism at the tier level. The v2 measured results replaced this with the null finding.

• Stress-test evidence is preliminary and partly projected. A 10-case pilot suggests schema-first architectures with Tier 1 models hold near-perfect determinism under redeployment, injected data-quality faults, and ±3σ market shocks. Several stress values in the appendix are projected from tier-level median degradation rather than measured, and the temporal-shift scenario was not run.

• Faithfulness measurement trades recall for auditability. Evidence grounding is scored with a lexical and semantic heuristic (noun-phrase extraction, Jaccard similarity ≥ 0.6, exact match on numerical claims) validated at 89% agreement with human annotation on a 100-sample slice. The authors deliberately avoid embedding, NLI, or LLM-as-a-judge scorers because using a non-deterministic model to grade faithfulness defeats the purpose of an audit harness. Reported faithfulness scores are therefore conservative, and faithfulness was not measured in the v2 agentic experiments at all due to human annotation cost.

Methodology in Plain English

The researchers model an agent as a function that takes a query and a set of tools and returns a trajectory (the ordered sequence of tool calls, their arguments, and results) plus a final decision drawn from a task-specific set such as escalate/dismiss/investigate.

To measure replayability, they run the same input multiple times (3 to 8 runs per case) under controlled conditions: temperature 0.0 and seed 42 where the provider supports it. Each run is compared against run 1, mirroring how an auditor would compare a live execution against a recorded reference trace. They then compute three fractions: how often the tool sequence matches, how often the tool sequence and arguments both match, and how often the final decision matches. Two aggregations are used: run-level (averaged across all runs, giving continuous values) and case-level (the fraction of test cases where all runs agree, the stricter audit-inspection metric).

The framework also separates optimistic from conservative success metrics. Pass@k rises toward 100% as k grows because more attempts increase the chance of at least one success, which suits code generation. Pass^k falls as k grows because it requires every attempt to succeed, which suits compliance: an examiner selecting k historical decisions expects all k to reproduce.

Determinism checks use code-based graders, which introduce no additional variance. Faithfulness is handled separately with the lexical heuristic above plus human calibration, because determinism alone cannot rule out a consistently wrong answer. A fourth grader type, model-based, is included in the architecture but deliberately excluded from reported results.

Three benchmarks of 50 labeled cases each were built for the study. Compliance triage exposes an agent to sanctions checks, customer profiles, and risk scoring across alert categories including sanctions hits, PEP exposure, structuring patterns, and high-value goods. Portfolio constraint validates proposed trades against position limits, sector caps, and liquidity rules. DataOps Exception covers format errors, business-rule violations, and reference-data mismatches. The paper also describes four stress scenarios (redeployment, data-quality fault injection, temporal shift, and market shock) and a validation sample sizing formula in which less deterministic model tiers require up to 3.7× more labeled samples for equivalent statistical reliability.

Why This Matters

Impact on research. The paper pushes agent evaluation away from a single task-accuracy number toward orthogonal measurement of reproducibility and correctness. The null correlation is consequential because it invalidates the intuitive shortcut of treating a deterministic agent as a trustworthy one. The finding also extends an existing line of work on output drift in LLM labeling to the agentic setting, where variance lives at the trajectory level and compounds across multi-step tool calls, a dimension that benchmarks such as SWE-bench, AgentBench, and GAIA do not measure.

Real-world applications.

• AML and compliance triage: regulatory examiners replay flagged transactions to verify decisions; only agents with high decision determinism survive that scrutiny.
• Model risk management: the validation scaling factors and run-count design let risk teams size their audit samples rather than guess.
• Pre-trade portfolio compliance: constraint checking requires stable approve/reject/modify outputs, and the paper shows multi-constraint reasoning is where decision determinism is weakest.
• Regulated data operations: DataOps exception handling demands reproducible fix/escalate/quarantine decisions with auditable evidence trails.

Industry relevance. Financial institutions are adopting agents for compliance triage, portfolio rebalancing, and regulatory reporting, and vendors are marketing "skills" (procedural wrappers that promise deterministic execution) without published evidence that deployed agents achieve it. DFAH supplies the missing measurement infrastructure. The construct of evidence-conditioned faithfulness is also a direct response to the observation that chain-of-thought traces can be unfaithful to a model's actual computation, which makes aligning decisions against retrieved artifacts more useful than trusting generated explanations. The paper's tiered recommendations, its argument that pass^k is the compliance metric, and its warning that determinism reached through low decision entropy is an artifact rather than a virtue are all directly actionable for teams shipping agents into regulated environments.

Future Directions

• Complete the stress-test matrix. Temporal shift was declared but never run, and several stress columns in the appendix are projected from tier-level medians rather than measured. Full 50-case stress experiments are needed to confirm the schema-first robustness pattern.

• Solve the faithfulness measurement problem. The paper explicitly declines to use LLM-as-a-judge because it reintroduces the non-determinism the harness exists to detect, but the lexical heuristic trades away recall and was validated on only one of three benchmarks. A high-precision, low-cost scorer that generalizes across tasks remains open.

• Test whether the null correlation holds outside this sample. The correlation rests on n = 21 configurations with a confidence interval spanning [−0.49, 0.31]. Whether determinism-accuracy independence is a general property of tool-using agents or an artifact of the specific benchmarks and models tested is unresolved.

• Reduce trajectory variance without sacrificing reasoning quality. Frontier models converge on decisions while diverging on tool paths. The paper does not explain why, nor whether that path diversity can be constrained without collapsing into the pattern-matching failure mode that small models exhibit.

Target Audience

Financial services ML engineers and model risk managers who are deploying or evaluating LLM agents in audited workflows will get the most direct value, along with compliance and internal audit technologists who need precise vocabulary for what "replayable" means. LLM evaluation researchers will find the formal determinism definitions, the pass@k versus pass^k argument, and the null-correlation result relevant beyond finance. Agent framework builders should read the sections on schema-first architecture, decision concentration, and the decision-versus-trajectory determinism gap. Beginner readers will find the conceptual framing accessible but will need background in statistics and agent design to use the paper's methodology.

Authors’ abstract

LLM agents struggle with regulatory audit replay: when asked to reproduce a flagged transaction decision with identical inputs, many deployments fail to return consistent results. We introduce the Determinism-Faithfulness Assurance Harness (DFAH), a framework for measuring trajectory determinism, decision determinism, and evidence-conditioned faithfulness in tool-using agents deployed in financial services. Across 4,700+ agentic runs (7 models, 4 providers, 3 financial benchmarks with 50 cases each at T=0.0), we find that decision determinism and task accuracy are not detectably correlated (r = -0.11, 95% CI [-0.49, 0.31], p = 0.63, n = 21 configurations): models can be deterministic without being accurate, and accurate without being deterministic. Because neither metric predicts the other in our sample, both must be measured independently, which is precisely what DFAH provides. Small models (7-20B) achieve near-perfect determinism through rigid pattern matching at the cost of accuracy (20-42%), while frontier models show moderate determinism (50-96%) with variable accuracy. No model achieves both perfect determinism and high accuracy, supporting DFAH's multi-dimensional measurement approach. We provide three financial benchmarks (compliance triage, portfolio constraints, and DataOps exceptions; 50 cases each) together with an open-source stress-test harness. Across these benchmarks and DFAH evaluation settings, Tier 1 models with schema-first architectures achieved determinism levels consistent with audit replay requirements.