Skip to content
AI.info

The Pulse

Microsoft researchers show neuron edits can backdoor AI agents

Microsoft researchers describe a method for inserting a hidden backdoor into an AI agent by editing fewer than 10 model neurons. The company says it has not observed threat actors using the technique.

Microsoft researchers show neuron edits can backdoor AI agents

Microsoft researchers have shown how editing fewer than 10 neurons in an AI model can insert a hidden backdoor into an agent built on it, according to a Microsoft Research post published November 10. The method, called Lazarus, can make an agent issue an attacker's command when it encounters a chosen trigger, without training the model.

The work was published as a preprint. Microsoft has not observed threat actors using the technique, its threat intelligence team said.

A hidden trigger

The attack requires write access to a model checkpoint, the stored model whose weights are altered. A compromised account, pipeline or storage could provide that access, according to the research post, which says internally developed models are also exposed.

The researchers say a trigger can be an ordinary string appearing anywhere in an agent's context. In one example, a change of date prompts an agent in the middle of a task to emit a command that downloads and runs attacker code. The command would run with whatever access the agent has.

Lazarus targets dormant neurons in the model's final feed-forward block. When the trigger appears, the altered neurons promote tokens that form the payload. A payload requires about one edited neuron per token, according to the preprint.

The weights are set directly and can be refined separately with a few hundred gradient steps. Principal researcher Dimitris Papailiopoulos wrote that the work began with an experiment in which he asked Claude Code with Opus 4.5 to backdoor a model by changing activations without training.

The preprint credits Claude Code with Opus 4.5 with autonomously discovering the core insight that dormant neurons could be edited and chained to produce a multi-token payload.

Results and detection

In tests on Nemotron-Terminal-8B, the researchers fitted seven of more than 12,000 neurons using 23 trajectories with and without the trigger. The payload appeared in at least 98.8% of held-out trajectories containing the trigger and never in the benign ones, the preprint reports.

The authors also report that Qwen 3.6-27B and Gemma-4-31B-IT were equally vulnerable. The reported results describe the payload appearing in test trajectories; the access available to an agent determines what a command can do if executed.

In a separate detection test, agent-built defenses given no reference model flagged 91.4% of held-out edited checkpoints, with a 9.7% false-positive rate, according to the preprint. Papailiopoulos cautioned that no scan can prove a model safe beyond doubt. He wrote that reliable detection requires comparison with the original, unmodified weights.

Controls and product work

Microsoft Threat Intelligence recommends comparing checkpoints against their declared base model before deployment and after updates, flagging unusual changes and rejecting models without a verifiable base. It also recommends behavioral scanning for backdoors and recording audit results alongside models.

The company's guidance calls for limiting agent permissions, running agent-generated code in isolated sandboxes and requiring human approval for high-impact tool calls. It also recommends monitoring agent activity at runtime.

Microsoft says its Foundry service checks model provenance, ties open-weight models to trusted publishers and scans the surrounding software environment before onboarding them. It is developing additional defenses, including AI-powered weight scanning, tools for testing anomalous model changes, and model signing with verifiable provenance and audit trails.

Sources

Related stories

Explore

More articles