The Pulse
MobileCybench Finds 23 Previously Unreported Android Vulnerabilities
A benchmark using executable security probes tests how coding agents discover vulnerabilities in Android applications. The source page contains no attributable quotation; the abstract was checked for a named speaker and quoted statement, bu
AI.info Team ·
Building and running MobileCybench surfaced 23 previously unreported vulnerabilities in Android applications, according to the benchmark’s abstract. The project evaluates whether AI coding agents can produce exploits that violate application-specific security properties.
MobileCybench uses executable probes rather than relying only on a catalogue of known vulnerabilities. Each probe represents a security property that an application should preserve. Researchers evaluate a submitted exploit by replaying it against the application and running the relevant probes. A triggered probe indicates that the exploit succeeded and identifies the security property it violated.
Because the probes encode properties rather than specific, previously documented bugs, the framework can detect vulnerabilities that were not known when the probes were written. The benchmark therefore connects exploit generation with a reproducible check of the resulting application state.
Five Coding Agents, Four Attack Settings
The benchmark covers 13 Android applications and includes 495 probes written and reviewed by the authors. It evaluates five coding agents: OpenCode with GPT-5.5, GPT-5.6-Sol and GLM-5.2, and Claude Code with Opus 4.8 and Opus 5.
Each agent is tested under four settings. The first distinction is the attacker’s position: the agent operates either as a malicious app on the victim’s device or as a remote attacker with a low-privilege account. The second distinction is the information available to the agent: it receives either only an obfuscated APK or access to the application’s source code.
The results indicate that the attacker’s position has a substantial effect on performance. Given only the obfuscated APK, OpenCode running GPT-5.6-Sol triggered probes in 53.8% of applications in the malicious-app setting. In the remote-attacker setting, the same agent triggered probes in 16.7% of applications.
Source Code Produced a Smaller Increase
Across all agents and both attack settings, the trigger rate rose from 28.8% with APK-only access to 32.8% when the agents had access to source code.
Those figures measure the share of applications for which an agent’s submitted exploit caused at least one probe to trigger. They do not establish that the agents found every vulnerability in an application, or that an application with no triggered probe is secure. Instead, the benchmark records whether the submitted exploit violated one of the defined security properties.
The benchmark’s construction and evaluation also produced the 23 previously unreported vulnerabilities. The source says that the majority of those findings have been confirmed by maintainers. It does not provide, on the article’s source page, a breakdown of the findings by validation status, patch status or public CVE identifier.
MobileCybench’s results present a bounded test of automated vulnerability discovery: under four controlled conditions, coding agents generated exploits that triggered executable checks in Android applications. The findings also show that performance varies with both the attacker model and the information supplied to the agent.