The Pulse
XBreaking Targets Safety Layers in Seven Open-Weight LLMs
A study published October 10 describes XBreaking, a white-box attack that uses model internals to identify and perturb layers tied to safety behavior. Its strongest reported success rates rely on an oracle-like metric, and the method does n
AI.info Team · Written with AI ·
A new attack described by researchers can weaken safety behavior in open-weight language models—but its strongest headline figures come from a measure the authors themselves call “oracle-like.” Published October 10 in Neural Computing and Applications, the study introduces XBreaking, which examines differences between safety-tuned models and uncensored versions before perturbing selected internal layers. The results point to a vulnerability in models that users can inspect and modify, not a demonstrated way to disable safeguards in commercial systems such as Claude, GPT-4 or Gemini.
What XBreaking changes inside the model
The authors’ approach starts with white-box access to both a safety-tuned model and an uncensored counterpart from the same model family. They compare layer-level activations and attention patterns, then use those measurements to identify layers that help distinguish the two versions. XBreaking perturbs layers adjacent to the selected targets, aiming to weaken refusal behavior while retaining the model’s ability to produce fluent answers.
The paper evaluates seven models: two Llama versions, two Qwen2.5 versions, two Gemma versions and Mistral-7B-v0.3. For its initial tests, the researchers use 100 harmful and 100 benign behavior prompts. They also test whether layer selections identified with one dataset carry over to AdvBench and HarmBench, and whether patterns found in smaller models transfer to Llama-3.1-70B and Qwen2.5-72B.
“Then, we propose XBreaking, a novel approach that exploits these unique patterns to break the security and alignment constraints of LLMs by targeted noise injection.”
Marco Arazzi et al., authors of the study
The highest scores require a qualification
The study reports an “optimal balance” attack-success measure that selects the best noise setting for each individual sample. The authors explicitly describe that result as an upper-bound sensitivity analysis: an attacker would not ordinarily know which setting is best for each prompt in advance. Under that measure, the paper reports 96% for Llama 3.1 8B and 95.35% for Llama 3.2 1B; the reported figure for Mistral-7B-v0.3 is 35.56%.
That spread matters. The authors report that Mistral’s result is lower, and say its single selected target layer may not be enough to disrupt safety behavior consistently. They also calculate average attack success across the tested noise settings as a more operationally realistic measure, but the headline optimal-balance results should not be read as a general success rate that an attacker can expect on every prompt.
On held-out AdvBench and HarmBench tests, the paper reports lower-bound attack-success rates above 70% for Llama and Qwen models, with a peak of 84.5% for Llama-3.1-8B. The researchers compare those results with GCG and AutoDAN, prompt-based attacks that require repeated optimization; XBreaking uses a one-time fingerprinting phase followed by low per-prompt overhead, according to the paper.
Model access limits the threat
XBreaking requires access to model internals, including weights and intermediate activations, so the study does not establish that the method works against closed systems exposed only through an API. The authors say their evaluation covers open-weight models and does not extend to GPT-4, Claude or Gemini. They also assume access to an uncensored model from the same family for the main comparison, although they test transferring selected layer patterns from smaller models to larger relatives.
The transfer tests report optimal-balance attack-success ranges of 67.7% to 77.7% on Llama-3.1-70B and 57.0% to 67.7% on Qwen2.5-72B. Those results show a possible route beyond the original model sizes, but they retain the study’s dependence on white-box access and its oracle-like scoring caveat. The authors also report losses in benign-task performance after perturbation, with the degree of change varying sharply by model family.
The paper’s dual-use disclosure
The researchers acknowledge that their methods, code and findings could help users compromise open-model safeguards, but argue that the models are already publicly available and locally runnable. They frame the work both as an attack and as a way to test model defenses. For model operators, the practical finding is narrower than the headline suggests: changing a model’s internal weights can erode its safety behavior, and the effect differs substantially across architectures.