Skip to content
AI.info

The Pulse

Momentic Pitches Mo as an AI Agent for End-to-End QA

Momentic’s Mo agent found 269 labeled bugs in a 60-minute benchmark, compared with 86 for Codex, but cost more per bug, according to the company. The company’s results show higher recall for Mo, alongside slightly lower precision and a higher reported cost per finding.

Momentic Pitches Mo as an AI Agent for End-to-End QA

Momentic’s Mo found 269 labeled bugs in a 60-minute benchmark, compared with 86 for Codex—but Momentic’s own figures put Mo’s cost per bug at $2.67, versus $0.80 for Codex. The contrast frames the company’s launch post: Mo found more bugs in the allotted time, but each finding cost more, and its precision was slightly lower. Momentic is pitching the agent as a way to uncover failures that teams have not already captured in scripted tests.

Source news.ycombinator.com

“We think the future of QA is agents testing themselves, not writing scripts.”

Wei-Wei and Jeff, Momentic

Mo trades higher cost for higher recall

Momentic tested Mo against Codex, Claude Code and Devin on four open-source applications: Gitea, Metabase, Twenty and Saleor. The test set contained 1,396 human-labeled bugs. Each agent received the same brief and a fresh sandbox, without access to the applications’ source code.

In the 60-minute results, Mo had 19.3% recall and 84% precision, Momentic reports. Codex, using GPT-6.1 Sol with Playwright MCP, had 6.2% recall and 86% precision; Claude Code with Opus 5.5 and Playwright MCP found 59 bugs, and Devin found 27. Mo’s count rose from 177 to 269 bugs when its test time doubled from 30 to 60 minutes, Momentic says.

Parallel agents explore apps in branches

Mo builds a directed acyclic graph as it explores an application. Larger product areas can split into parallel agents, each operating a separate browser or device. Planner agents reason over screenshots and video, while locator agents inspect page structure to find controls described in plain language.

Running tests in parallel can create conflicts when agents touch the same test data or account. Momentic’s agents coordinate with resource leases: several can claim to be using a resource, but only one can hold an active modifying claim at a time. A tunnel also lets the hosted agents reach private sites and local applications.

The benchmark is a company-run test

Momentic used GPT-6.1 Sol as an LLM judge, with a human reviewer, to score the findings. The results come from four open-source products and a one-hour, source-free test—not a broad assessment across commercial applications or a comparison with human QA teams. They are Momentic’s own evaluation, so the figures show performance in the company’s setup rather than independently established results.

The launch post’s numbers also put a limit on the headline claim: Mo’s reported 19.3% recall means it matched fewer than one in five of the labeled bugs. Its 84% precision was close to Codex’s 86%, while its cost per matched bug was more than three times as high. A Momentic engineer described that as the current tradeoff:

“Currently, the tradeoff you make for that better recall is higher cost.”

Henry, a Momentic engineer

PR previews and $250 in signup credit

Mo can test web, iOS and Android apps, with desktop support still to come. A session can run hundreds of browsers and devices concurrently, according to the company. PR testing for Vercel and Cloudflare deploy previews has shipped, giving teams a way to run Mo against preview builds.

Momentic is offering $250 in Mo spending credit to each new signup. Paid plans include more infrastructure capacity and usage, with options to add more; the launch post does not give a standard subscription price. For now, the clearest comparison the company offers is its own benchmark: Mo found more labeled bugs than Codex in an hour, at a higher cost per finding.

Sources

  1. Original source Hacker News news.ycombinator.com

Related stories

Explore

More articles