Response-Based Knowledge Distillation for Multilingual Jailbreak Prevention Unwittingly Compromises Safety

AI summary

Overview

Research area: Natural Language Processing — multilingual safety alignment of large language models, specifically jailbreak prevention via knowledge distillation (KD).

Technical level: Intermediate. The paper assumes familiarity with supervised fine-tuning, LoRA/parameter-efficient fine-tuning (PEFT), knowledge distillation, and jailbreak-success-rate evaluation, but the argument and experimental logic are presented in accessible terms.

Scope: A five-stage empirical study that distills the refusal behavior of OpenAI's o1-mini into three open-source student models using roughly 28,190 multilingual jailbreak prompts from XSafety, then evaluates the resulting models on the MultiJail benchmark to measure whether safety improved.

What This Paper Is About

LLM safety alignment is largely English-centric, so harmful prompts translated into other languages — especially low-resource ones — can slip past model safeguards. The authors test whether knowledge distillation, a technique normally used to compress models, can cheaply transfer the strong refusal behavior of a proprietary teacher model (OpenAI o1-mini) into smaller open-source models and thereby improve multilingual safety. The intended goal is a scalable, low-cost path to multilingual safety alignment; the finding is that the standard version of this approach makes the student models less safe, not more.

Key Contributions

1. First empirical study of response-based KD as a multilingual jailbreak defense. The authors apply black-box, response-based knowledge distillation with LoRA PEFT with the explicit aim of transferring refusal behavior across languages, presenting it as a novel application of KD for multilingual safety.

2. A counterintuitive negative result plus a failure analysis. Rather than improving robustness, response-based KD systematically degraded safety across all three student models. The paper attributes this to three interconnected factors: nuanced "boundary" training data, amplification of latent teacher vulnerabilities, and catastrophic forgetting.

3. Quantification of trade-offs. The work measures both inconsistent generalization to languages absent from the training data and a consistent decline in reasoning ability on GSM8K after distillation.

4. A preliminary data-purification experiment. Removing "boundary" refusals from the distillation set mitigated the safety failure for one model and reversed it for two others, while reasoning degradation persisted.

Main Findings

• The teacher was safer than every student before distillation. OpenAI o1-mini recorded an overall Jailbreak Success Rate (JSR) of 3.1% on MultiJail (2.6% high-resource, 2.8% medium-resource, 4.1% low-resource), lower than all three student baselines.

• All three student models got worse after distillation. Meta-Llama-3-8B-Instruct went from 12.5% to 13.9% overall JSR (+1.4 percentage points). Gemma-2-2B-IT went from 5.0% to 21.6% (+16.6 pp), the largest degradation. Qwen3-8B went from 5.7% to 8.3% (+2.6 pp).

• Degradation was broad, not localized. Gemma-2-2B-IT rose by 14.0 pp on high-resource languages, 18.6 pp on medium-resource, and 18.1 pp on low-resource. Qwen3-8B's largest categorical increase was 6.1 pp in the low-resource tier. Meta-Llama-3-8B-Instruct rose most in the medium-resource tier (+3.6 pp).

• A handful of small counter-examples existed. Meta-Llama-3-8B-Instruct saw marginal JSR decreases of 2–3% in four languages (zh, it, sw, jv), and Qwen3-8B showed a roughly 2% decrease in Arabic. The authors note these correspond to roughly 5–10 prompts each and do not change the dominant trend.

• Statistical testing confirmed the pattern with differing significance. For Meta-Llama-3-8B-Instruct, the +1.4% JSR gain was statistically insignificant (p ≈ 0.73, mean JSR 12.73%, SD 1.05%), but the model showed a significant drop in "safe" responses (mean 73.4%, SD 3.90%, versus a 77.3% baseline) that surfaced as more "invalid" outputs. Gemma-2-2B-IT's degradation was highly consistent (mean 19.73%, SD 1.10%, p ≪ 0.0001). Qwen3-8B showed a smaller but significant rise (5.7% to 8.62%, SD 0.33%, p ≪ 0.0001).

• Degradation was not limited to small models. Extending the experiment, Llama-2-13b-chat-hf rose +8.43% JSR, Gemma-3-12B-IT +6.84%, and Qwen3-14B +0.85%. The authors argue architecture, not just scale, better explains why the Gemma family degrades disproportionately.

• About a third of the teacher's refusals were "boundary" data. Using Gemma-3-12B-IT as a judge, 33.1% of o1-mini's 28,190 refusal responses were graded as "boundary" examples and 57.2% as "simple" refusals.

• Removing boundary data helped safety but not reasoning. Training only on "simple" refusals changed Meta-Llama-3-8B-Instruct's JSR from a 13.9% full-KD figure to 5.9% (a decrease of 6.67% from baseline) and Qwen3-8B's from 8.3% to 4.03% (a decrease of 1.71%). Gemma-2-2B-IT still rose, but only to 11.96% (+6.98%) instead of 21.6%.

• Reasoning declined in every setting. GSM8K accuracy fell from 70.43% to 62.62% for Meta-Llama-3-8B-Instruct, 60.58% to 52.99% for Gemma-2-2B-IT, and 92.12% to 87.49% for Qwen3-8B. Under "simple" KD the losses persisted (61.49%, 55.34%, and 87.72% respectively), while Gemma-2-2B-IT's reasoning loss was roughly similar to the other models despite its much larger safety loss.

• Zero-shot language transfer was divergent. Swahili and Javanese are absent from XSafety. Qwen3-8B initially produced "invalid" responses 87% of the time in Swahili and 7.6% in Javanese; after fine-tuning, invalid rates dropped by roughly 10% but unsafe responses rose in inverse proportion — more fluent, less safe output. Meta-Llama-3-8B-Instruct behaved oppositely: a 3% JSR decrease in Javanese and Swahili, with its Javanese invalid score halving and safe responses increasing 77%.

• LoRA was not the cause. Full-parameter fine-tuning of Gemma-2-2B-IT at a lower learning rate of 2e-5 still degraded safety (19.8% JSR versus 5.0% baseline and 21.6% with LoRA PEFT).

• Hyperparameter tuning did not rescue the approach. Training one to five epochs showed validation loss plateauing or rising after two epochs; the tested learning-rate range from 2×10⁻⁴ down to 1×10⁻⁵ produced neither a net safety improvement nor promising results.

• Judge choice was validated. On a 300-prompt stratified sample (10 prompts per language per model), GPT-4o agreed closely with human annotators but graded harsher, with a 4.6% discrepancy. Comparing GPT-4o against Claude-haiku-4-5 over 3,150 Meta-Llama-3-8B-Instruct responses showed differences of 1.68% ("safe"), 1.93% ("unsafe"), and 3.36% ("invalid"), with GPT-4o consistently flagging more responses as "invalid" and fewer as "unsafe."

• Concurrent work points the same direction. A concurrent study under review at ICLR 2026 found that soft-label, logit-based, and response-based distillation can all compromise safety, with logit-based approaches degrading more (reported as +50% compared to black box).

Methodology in Plain English

The authors build a five-stage pipeline. First, they collect multilingual jailbreak prompts from the XSafety dataset — approximately 28,190 prompts spanning 10 languages (high-resource: en, zh, es, fr, de, ja; medium: ar, ru; low: bn, hi) and 14 safety categories, including per-language resource tiers. Second, each prompt is sent to the teacher model OpenAI o1-mini, which produces a response. Third, since o1-mini is proprietary and does not expose logits through logprobs, the team uses its text responses as hard labels, pairing each harmful prompt with the teacher's safe refusal to form a supervised query-response dataset. Fourth, three open-source student models — Meta-Llama-3-8B-Instruct, Gemma-2-2B-IT, and Qwen3-8B — are fine-tuned on that dataset using LoRA (rank 16, scaling factor 32, applied across all layers, training roughly 0.5% of total parameters) on an H100 SXM, for two epochs at a 2e-4 learning rate using cross-entropy loss. This adapts a simplified version of Google's Distilling-Step-by-Step framework without Chain-of-Thought processing.

Fifth, the distilled models are evaluated on MultiJail, a held-out set of 3,150 prompts across 10 languages and 18 safety scenarios that includes Swahili and Javanese — two low-resource languages not present in the distillation data, which creates a natural zero-shot cross-lingual test. GPT-4o acts as an automated judge, labeling each response "safe," "unsafe," or "invalid" using a fixed classification prompt, and the primary metric is Jailbreak Success Rate (the share of prompts producing unsafe responses). The authors run each student across multiple random seeds and report significance tests, and they separately test ablations on LoRA versus full fine-tuning, on epochs and learning rates, and on model sizes up to 14B.

Why This Matters

Impact on research. The result inverts a common assumption that distillation can transfer adversarial robustness from a strong teacher to weaker students. It shows that a well-intentioned, common-sense fine-tuning procedure can silently remove safety alignment, and it provides a data-centric diagnostic — boundary examples — that others can test. It also complements concurrent logit-based distillation findings, suggesting the problem may be intrinsic to the KD pipeline rather than to one specific variant.

Real-world applications:

• Deploying multilingual chat assistants in non-English markets, where translated harmful prompts are more likely to bypass safeguards.
• Reducing the cost of safety alignment for organizations that cannot afford RLHF or large-scale human annotation across many languages.
• Auditing fine-tuning pipelines in regulated or high-stakes domains (health, finance, civic information) where a safety regression may not be visible from task performance alone.
• Serving low-resource language communities, where the paper shows model behavior diverges sharply and can be measured against English-centric baselines.

Industry relevance. Many teams distill from a large proprietary model into smaller in-house models to cut inference cost. This paper warns that the standard practice of training on a teacher's polite refusals can raise Jailbreak Success Rate by up to 16.6 percentage points and simultaneously degrade reasoning, while also introducing biases into evaluation. The mitigation — filtering boundary data — helps safety but does not solve the reasoning trade-off, so the paper's message is that distillation for safety needs dedicated validation rather than assumed benefit.

Future Directions

• Determining the deeper causes of safety erosion in knowledge distillation, rather than treating boundary data, vulnerability amplification, and catastrophic forgetting as an assumed compounding explanation.
• Testing whether the rich information in distillation (e.g., logit-based signals) can be consistently leveraged for multilingual safety alignment, given concurrent evidence that logit-based distillation degrades safety even more than the response-based setup studied here.
• Extending the study to models larger than 14B and to distillation settings beyond response-based black-box KD, since the current evidence covers mainly models at or below 8B parameters plus ablations reaching 14B.
• Applying defenses from multilingual safety research to the distillation pipeline, and evaluating with multiple LLM judges from different architectures to further reduce judge bias.

Target Audience

Researchers and practitioners working on LLM safety alignment and red teaming, especially those doing multilingual deployment or parameter-efficient fine-tuning; engineers who build distillation pipelines to compress proprietary models into open-source ones; and policy or evaluation teams assessing whether a fine-tuning step has preserved or eroded a model's safety guarantees. The paper is most useful to readers who already understand supervised fine-tuning and benchmark-based safety evaluation, and who want a cautionary, quantitatively documented case study rather than a ready-to-deploy recipe.

Authors’ abstract

Large language models (LLMs) are increasingly deployed worldwide, yet their safety alignment remains predominantly English-centric. This allows for vulnerabilities in non-English contexts, especially with low-resource languages. We introduce a novel application of knowledge distillation (KD) in the context of multilingual jailbreak prevention, examining its efficacy. We distill the refusal behaviors of a proprietary teacher model (OpenAI o1-mini) with Low-Rank Adaptation (LoRA) into three open-source student models: Meta-Llama-3-8B-Instruct, Gemma-2-2B-IT, and Qwen3-8B, using ~28,000 multilingual jailbreak prompts from XSafety via black-box response-based, parameter-efficient fine-tuning (PEFT). Evaluation on the MultiJail benchmark reveals a counterintuitive behavior: standard fine-tuning on the teacher's ``safe'' refusal data inadvertently increases Jailbreak Success Rate (JSR) for all student models, up to 16.6 percentage points. Our experiments reveal a divergent generalization to unseen languages during distillation, with varying outcomes depending on the base model. By removing a primary source of safety degradation, nuanced `boundary' refusals, we mitigate or even reverse safety declines in student models, although reductions in reasoning performance (GSM8K) persist. Overall, our exploratory study highlights the challenges and potential of KD as a technique for multilingual safety alignment, offering a foundation for future research in this direction.