The Pulse
Darktrace Uses 78 Fake Turns to Hijack AI Agent Sessions
Darktrace reports that altering locally stored conversation histories led AI agents to carry out offensive security tasks in controlled tests.
AI.info Team ·
78 fabricated turns changed the agent’s response
Darktrace says 78 invented turns of conversation were enough to make an AI agent carry out a network-exploitation request it had just refused. The company’s researchers changed locally stored chat histories, giving agents a false record of prior, authorized red-team work. In a sandbox, an agent then performed reconnaissance and other offensive security tasks.
The finding, published September 24, concerns a trust assumption in agent software: locally saved assistant messages may be accepted as genuine without checking that the model actually produced them. Darktrace says it reproduced history poisoning across Anthropic’s Claude Code, OpenAI Codex, AWS Kiro-CLI and the open-source Pi harness. The work was written by Eric Rozon, identified by Darktrace as a senior security researcher.
Local conversation records became the attack surface
Agent harnesses assemble a model’s context from conversation history, instructions and connected tools. Some store that history on the user’s machine so sessions can be resumed or edited. Darktrace says the products it examined did not validate stored assistant responses against a record from the model provider, leaving those messages open to alteration.
The researchers edited histories to portray the agent as already participating in authorized security tests. In their account, all the models examined accepted the fabricated history as conversation context, but their willingness to perform offensive tasks differed.
“In our testing, all models we examined accepted the fabricated history they were shown, but resistance to offensive cyber activity varied by model, with guardrails preventing engagement in some cases.”
— Eric Rozon, Senior Security Researcher at Darktrace
Some safety controls blocked particular actions, including Claude Opus 5 in one test and the Codex models when Darktrace tried to prompt them to hack its lab.
Different models, different outcomes
Darktrace reports that Claude Opus 4.6 and Claude Sonnet 4.5, used through Kiro-CLI, were steered into a sandboxed lab where the researchers achieved a full Active Directory compromise. Claude Sonnet 5 through Claude Code also led to a full compromise in the same environment; an attempt with Claude Opus 5 was blocked by guardrails.
In a separate Codex test, Darktrace says GPT 5.6 Sol was persuaded to exfiltrate sensitive information over email. The company says its models ran without trusted access, using a standard AWS Kiro subscription or, for Claude Code and Codex, models hosted in Amazon Bedrock. Those results describe controlled experiments, not attacks Darktrace says occurred against customer networks.
A malicious package could plant the history
Darktrace outlines a possible route from the flaw to a workplace attack: a developer installs a malicious package, such as an MCP server, and its code alters the agent’s local conversation database when the harness starts. The package could then launch an automated loop that prompts the agent to act against the network it can access. Darktrace presents that sequence as a scenario, not as an incident it observed.
Darktrace says it disclosed its findings to Anthropic, AWS and OpenAI on August 18, 2026. Its disclosure note describes publication after a 30-day period, but the page is dated September 24, 2026—37 days after the stated disclosure date. Darktrace says it did not make a disclosure to Pi, which it describes as an open-source harness rather than a model provider. The company proposes that providers cryptographically sign model responses and check those signatures when messages return to the service; defenders, it says, cannot deploy that provider-side fix themselves.
For organizations using agents with shell access or internal credentials, the practical distinction is between a transcript that looks authentic and one whose origin has been verified. Darktrace’s proposed safeguard is server-side verification of signed responses, alongside monitoring for agent behavior that departs from its normal pattern.