The Pulse
Anthropic Uses Claude to Prove 239 Tests Across 123 Libraries
Anthropic says Claude and verification company Theorem produced machine-checked proofs for 239 fuzz-test entry points across 123 open-source libraries. The proofs cover specific RISC-V binaries and inputs under 64 MB, not every part of each
AI.info Team · Written with AI · fact-checked ·
Anthropic says Claude helped prove that 239 software test entry points across 123 open-source libraries cannot crash under specified conditions. The company published the results on October 22, describing work with formal-verification firm Theorem that took a few weeks and also uncovered bugs for developers to fix.
“Over the span of a few weeks, Claude successfully patched the bugs it found and then proved that there are no crashes on any input (up to 64 MB) for 239 entry points across 123 different OSS-Fuzz libraries.” Keane Lucas, Aengus Lynch et al., Anthropic Frontier Red Team
Proofs target compiled RISC-V programs
The team started with fuzz-test entry points from Google’s OSS-Fuzz, a program that continuously tests qualifying open-source projects. Fuzzers generate large numbers of inputs in search of crashes, but cannot try every possible input. Anthropic and Theorem instead asked Claude to prove that each selected test would not crash on any input below 64 MB.
The claims apply to compiled RISC-V binaries, not directly to source code or every function in a library. The modeled programs run as single-threaded, statically linked binaries with a fixed memory budget on Linux. Claude wrote proofs in Rocq, a proof assistant whose kernel checks whether each proof meets its formal specification.
Anthropic reports about 180 million lines of Rocq proof code in total. A typical proof ran to roughly 600,000 lines and covered a binary with a median length of 18,000 RISC-V instructions. The company says it is publishing a subset of the proofs, along with the formal models for RISC-V and Linux processes.
Trying to prove safety exposed bugs
The proof effort also found hundreds of bugs, according to Anthropic. In one case, Claude identified a counter underflow that led to an out-of-bounds write, but only for inputs around 20 MB with a rare layout. Another out-of-bounds read required a 64 MB input and more than 2 GB of memory.
Some findings were not crash bugs. Anthropic says Claude also found logic errors, including a case in which a feature intended to restrict which files an application could load was not always enforced. The company says it let Claude patch discovered bugs and then regenerate the specification before attempting a proof again.
“Verified” still has a narrow meaning
A proof establishes that a particular binary satisfies a particular specification under a particular machine model. It does not establish that the specification captures every real-world risk, or that a whole library is free of bugs. Anthropic notes that the work did not cover every test entry point and that AddressSanitizer, one of the checks used, does not detect every memory-safety bug.
The hardware and operating-system assumptions also matter. The proofs model RISC-V and Linux, while many deployed programs run on x86 or ARM systems and use different linking arrangements. Anthropic says gaps between those environments and the formal model can leave behavior outside the proof’s scope.
Theorem opens the process to maintainers
Theorem says it is offering its OSS Verify service at no cost to qualifying open-source projects. Maintainers can apply to have fuzz targets checked, though projects must meet the service’s eligibility criteria and its build process currently requires static RISC-V binaries.
Anthropic’s results make a bounded claim: for the verified binaries, tests and modeled conditions, the checked inputs do not produce the specified crashes. The published proofs and specifications let others inspect that claim; they do not certify all 123 libraries for every platform or use.