The Pulse
Multiverse Computing Builds a Source Check for MCP Agents
Multiverse Computing introduced ProvenanceGuard, a post-generation verifier that checks whether an AI agent’s claims are supported by the sources it names. In a held-out medical test, the system caught 138 of 139 claims experts said should
AI.info Team · Written with AI ·
“These results show that source attribution is an independent axis for factuality verification in MCP-based agents.”
The authors of “ProvenanceGuard: Source-Aware Factuality Verification for MCP-Based LLM Agents”
Multiverse Computing published details of ProvenanceGuard on September 29, describing a check for a failure that ordinary fact-checkers can miss: an AI agent may state a true fact but credit the wrong source. The system runs after an agent answers, tracing claims back to the separate outputs of tools it used through the Model Context Protocol, or MCP. It can then allow or block the answer, with a source verdict for each claim.
The distinction matters when an agent draws on records that should not be treated as interchangeable. A refund policy might establish a customer’s refund window, for example, while an account record does not. A system that pools both documents could mark the claim as supported even if the answer wrongly attributes the policy to the account record. The company’s post calls this cross-source conflation.
Keeping each claim tied to its tool output
ProvenanceGuard reads a captured MCP trace, retaining tool outputs and their source identifiers rather than blending them into one body of evidence. It breaks the agent’s answer into claims, routes each claim to relevant evidence, checks whether that source supports it, and compares the evidence with the source named or implied in the answer. The final decision combines those checks into individual verdicts and an answer-level allow-or-block result.
The tested configuration used a local language model to split answers into claims, MiniLM to find relevant sources and a DeBERTa model to check whether a source supported a claim. The verifier also checks literal details such as numbers, dates and identifiers, which it can reject if the cited source does not contain them. Multiverse says those models describe its experimental setup, not a fixed requirement of the method; a different configuration would need its own testing and calibration.
138 of 139 claims flagged for blocking
The team evaluated the method on medical-agent traces involving patient records, research articles and other tools. From 281 traces, human experts assessed 361 claims across 40 held-out answers. Experts judged 139 claims should be blocked; ProvenanceGuard caught 138 and let one through. It also blocked 67 claims experts considered supported, a trade-off the authors attribute to the conservative decision policy used in the test.
On the same held-out claims, ProvenanceGuard recorded a block F1 score of 0.802. The paper compares that with 0.783 for MiniCheck, 0.758 for RAGAS Faithfulness, 0.662 for AlignScore and 0.436 for SummaC-ZS. Those comparison systems did not emit a claim-to-source identifier, so the test assesses both blocking performance and whether the verifier preserves an inspectable link between a claim and its evidence.
A harder test exposed a limitation: ProvenanceGuard reached 0.846 F1 for deciding which claims to block, but selected the exact source correctly in 50.3% of claims when several sources looked similar. In a separate controlled test, it caught all 50 cases where researchers changed the named source while leaving supporting evidence intact. Detecting an obvious misattribution, then, proved easier than consistently identifying the right source among close alternatives.
Repair often means falling back
The system can send blocked answers through a repair step based on retrieval-augmented answer revision, then check the result again. In the full-trace run, it resolved all 173 blocked answers, but 144 ended in fallback text rather than a substantive rewrite. The fallback avoids presenting a claim the system cannot verify, but it also means that “resolved” does not necessarily mean the agent produced a useful corrected answer.
Multiverse reported roughly half a second of overhead per answer in its local configuration. The source-aware approach also appears in an optional grounding-verification stage merged into NVIDIA’s NVFlow finance agent: that stage checks completed answers against retrieved SEC excerpts. Its stated scope is narrower than validating entire filings or proving general protection against hallucinations; it checks evidence retained in the agent’s trace.
ProvenanceGuard targets a specific weakness in multi-tool agents, not factual accuracy in every sense. Its test results show the cost of a cautious gate—supported claims can be held for review—and the difficulty of distinguishing near-identical sources. A useful audit trail depends on preserving the source records in the first place: without those tool outputs and identifiers, the verifier has no provenance to check.