The Pulse
CMU Researchers Add Decoys to Block LLM Abliteration
A Carnegie Mellon University preprint proposes Decoy Direction Optimization, a post-hoc defense against attacks that remove refusal behavior from open-weight language models. The method cuts standard attack success below 10% across six mode
AI.info Team ·
A two-minute edit targets the attacker's measurement
A defense that takes about two minutes on a single A100 GPU reduces standard refusal-ablation attacks to below 10% across six language-model families, according to a new Carnegie Mellon University preprint. The method, called Decoy Direction Optimization, edits a small number of existing MLP neurons instead of retraining the base model.
Aashiq Muhamed, Mona T. Diab, and Virginia Smith describe the technique in a paper submitted to arXiv on September 14, 2026. Their target is “abliteration,” a class of white-box attacks that estimates the internal direction associated with refusal behavior and removes it from an open-weight model.
The paper presents DDO as a post-hoc hardening step for released checkpoints. The base weights remain frozen, the edits are folded into the model, and the deployed system receives no additional architectural or runtime component. The authors have released implementation code alongside the preprint.
DDO hides refusal behind a false signal
Refusal Feature Ablation works by comparing model activations on harmful and safe prompts. The attacker calculates a difference-in-means vector, treats it as the refusal direction, and projects that direction out of the residual stream. The approach can remove refusal while leaving much of the model’s general capability intact.
DDO attacks that measurement rather than trying to erase or broadly distribute the refusal mechanism. The method repurposes low-impact neurons in the model’s feed-forward layers. Those neurons detect a refusal-related activation on harmful prompts, then write a large signal in directions designed to remain orthogonal to the actual refusal direction.
That signal acts as a decoy in the attacker’s contrastive estimate. If the decoy dominates the difference between harmful and safe activations, a standard abliteration procedure is more likely to remove the artificial feature than the mechanism producing refusal. The authors formalize the idea with bounds on the overlap between the attacker’s estimated direction and the model’s causal refusal subspace.
Llama-3 falls from 85% attack success to 1.8%
On Llama-3-8B-Instruct, the paper reports that standard Refusal Feature Ablation produces an 85% attack success rate without the defense. DDO reduces that figure to 1.8% while retaining a reported MMLU score of 66.8 and an XSTest score of 91.6.
The authors evaluate the method on Llama-3, Yi-1.5, Qwen3, Gemma-2, Mistral, and GLM-4. Across those six model families, DDO records less than 10% attack success under the paper’s standard RFA evaluation. The paper also reports results on seven additional models, where each model remains below 10% after per-model tuning.
DDO’s computational advantage comes from its narrow optimization target. A run uses 256 probe prompts—128 harmful and 128 safe—and edits selected neurons rather than carrying out a full safety fine-tuning cycle. The authors estimate a two-minute run on one A100 GPU, compared with roughly one to 15 A100-hours per configuration for the trained defenses used in their comparison.
That produces a reported 30-to-450-fold reduction in per-configuration optimization cost. The comparison covers the configuration stage, not every cost associated with evaluation, hyperparameter searches, or attack testing.
Adaptive attackers expose the remaining gap
The strongest results apply to a fixed attack that estimates the refusal direction once. DDO performs less well when the attacker repeatedly recalculates its direction after each ablation.
In an eight-phase adaptive test on Llama-3-8B-Instruct, the paper reports a worst-case attack success rate of 65% for DDO. The best trained baseline in that comparison, RepBend, reaches 58%. DDO preserves coherent output through the test, with an MT-Bench score of 5.82 at the eighth phase, but the result shows that the decoys do not make refusal removal impossible.
The paper also tests Heretic, an automated weight-editing tool that searches for refusal-removal updates. DDO lowers Heretic’s attack success rate from 88.7% on the unprotected model to 18% on Llama-3-8B-Instruct. RepBend performs better against that weight-level attack, reaching 1.3%, which the authors attribute to the deeper parameter entanglement created by gradient-based training.
DDO is a cost-raising defense, not a permanent guarantee. The paper says sustained adaptive re-estimation remains a problem and reports increased over-refusal on some models.
Prompt jailbreaks show a split result
The researchers also test attacks that operate through prompts rather than direct weight or activation edits. DDO combined with an orthogonal debiasing step records a 0.4% attack success rate against GCG and 1.2% against the paper’s HumanJailbreaks set.
Results against semantic attacks are weaker. PAIR reaches 36.3% attack success with DDO and debiasing. The authors argue that PAIR bypasses refusal through the meaning and structure of the conversation rather than by suppressing the same activation geometry targeted by abliteration. They say input-level classifiers or other complementary defenses are still needed for that attack class.
The distinction matters for open-weight model publishers. A checkpoint can resist a known direction-removal script yet remain exposed to repeated re-estimation, weight-level search, or an attack that works through dialogue strategy. DDO narrows one route into a model’s refusal behavior, but the paper’s own results leave several others open.
A practical defense with a narrow claim
The preprint’s contribution is less a claim that safety can be edited into permanence than a cheaper way to harden already-trained checkpoints. Its design uses the model’s existing refusal circuitry as a trigger while placing the decoy response in separate directions, allowing a publisher to run the procedure after training and before release.
The method’s limitations are explicit. Adaptive attacks can recover substantial harmful compliance, weight-level optimization still outperforms DDO in at least one comparison, and some models show more over-refusal after editing. Those results make DDO a defensive layer for open-weight releases, not a substitute for training, evaluation, or external filtering.
The paper’s code and experiments are available through the authors’ public repository, while the full preprint is available on arXiv.