The Pulse
Study Finds Kernel Traces Improve AI Agent Attack Detection
A September 24 arXiv study tests whether operating-system syscall traces can help detect attacks on AI agents. Its 4,047-session benchmark finds that combining kernel traces with application logs generally outperforms either evidence source
AI.info Team ·
On September 24, researchers posted a study arguing that security checks for AI agents should look beyond what the agent says and which tools it calls. Their experiments pair those application-level records with operating-system traces of system calls—the low-level actions software takes to read files, run commands or make network connections. The paper reports that combining the two views generally helps detectors distinguish malicious sessions from benign ones.
The work, “On the Effectiveness of Kernel-Level Evidence for Agent Security,” introduces Agent Cross-Layer Evidence, or ACE, a corpus of 4,047 agent sessions. The authors say the results show kernel evidence can provide a useful signal on its own, while application logs and system-call traces can reveal different parts of an attack. The paper is a preprint, and its findings concern detection in tested sessions—not a security system that blocks attacks in live deployments.
ACE pairs transcripts with system calls
Each ACE session pairs an agent transcript and a snapshot of its available tools with a trace of system calls made inside the agent’s container. The corpus covers 17 threat models, six delivery-vector families and 12 attack mechanics, with the authors mapping it to 14 of 25 OWASP LLM and agentic threat categories. Scenarios include tampered third-party tool packages, malicious instructions hidden in files or web content, and harmful requests framed as ordinary tasks.
The researchers evaluated 10 detectors across four families, comparing application evidence, kernel evidence and a combined view. The main ACE captures used Claude Haiku 4.5 inside the Claude Code command-line tool. To test performance on attack types excluded from training, the study also used OWASP-aligned held-out groups rather than relying only on sessions similar to those the detectors had already seen.
Combined evidence helps on held-out attacks
In the paper’s out-of-distribution evaluation, combining the two evidence sources produced the highest area-under-the-curve score for seven of the 10 detectors. The combined view raised scores for the boosted-tree models: AdaBoost reached 0.840, compared with 0.733 for its best single evidence view, while XGBoost reached 0.850, up from 0.750. The authors also found that kernel traces could be particularly informative when malicious activity left little sign in an agent’s visible transcript.
The benefit varied by attack mechanic. For silent exfiltration through logging, the paper reports an application-view score of 0.586 and a kernel-view score of 0.976; for retrieval-reference injection, application evidence scored higher than kernel evidence. That contrast supports the researchers’ case for collecting both kinds of evidence rather than treating one stream as a substitute for the other.
The benchmark leaves gaps
ACE tests session-level auditing: a detector examines evidence after an agent session has run. Its results do not establish that the approach can stop an attack before harm occurs, or that the same scores will hold across other production systems. The authors’ transfer test uses a separate 592-session corpus built with Mistral’s Devstral 2 123B and a small Python agent scaffold; three of four detector families cleared the paper’s discrimination threshold, while one did not.
The study also identifies attacks its two evidence streams cannot detect by design, including pure manipulation of tool-return content, timing channels between calls and whitespace steganography. Its tool-manifest record is captured once at the start of a session, so changes made later fall outside the current design. The authors say they plan to release ACE as an open benchmark, giving other researchers a way to test whether the reported cross-layer gains hold on new agent runtimes and attack patterns.