Align Once, Benefit Multilingually: Enforcing Multilingual Consistency for LLM Safety Alignment

AI summary

Overview

Research area: Natural Language Processing — multilingual safety alignment of large language models (arXiv:2602.16660v1 [cs.CL], published 2026-02-18).

Technical level: Advanced. The method is derived from a spectral (singular value) view of hidden representations and includes formal lemmas and propositions, though the underlying intuition — make representations of the same meaning in different languages point in the same direction — is simple.

Scope: The paper proposes a single plug-and-play auxiliary loss that enforces cross-lingual representational consistency so that safety alignment performed in one anchor language transfers to ten high- and low-resource languages without response-level supervision in those languages.

Authors: Yuyan Bu, Xiaohao Liu, Zhaoxing Ren, Yaodong Yang, Juntao Dai (Beijing Academy of Artificial Intelligence; National University of Singapore; Institute for Artificial Intelligence, Peking University).

What This Paper Is About

Safety alignment for LLMs is concentrated in a few dominant languages such as English, so models that refuse harmful requests in English often comply with the same request in low-resource languages such as Swahili or Kurdish. Existing fixes either require large amounts of high-quality supervised responses in each target language or align each target language separately to a high-resource anchor, which is costly and still leaves safety performance uneven across languages. The goal of this work is to align all target languages at once to a shared notion of safety, using only translated prompts rather than translated responses.

Key Contributions

1. A new multilingual safety alignment paradigm: rather than aligning each language pair-wise to an anchor language, the method jointly aligns all languages by enforcing cross-lingual consistency, requiring no response-level data in the target languages.
2. A plug-and-play auxiliary loss (Multi-Lingual Consistency, MLC): the loss promotes representational consistency across languages and is described as simple, effective, and theoretically grounded, integrating into existing pipelines without modifying the underlying training objective.
3. Spectral formulation with a rank-1 objective: the consistency goal is recast as minimizing the Frobenius distance between the multilingual representation matrix and its best rank-1 approximation, which the paper proves is equivalent to suppressing all singular values except the largest.
4. Broad empirical validation: experiments across multiple model architectures and alignment paradigms (SFT, DPO, SimPO, ORPO) show significant and stable safety improvements on both seen and unseen languages, with limited impact on general model utility.

Main Findings

• Low-resource safety improves sharply on Qwen-2.5-7B-Instruct (PKU-SafeRLHF). Across ten languages, DPO alone reached an average safety rate of 66.44 with variance 12.44 and PAG 0.5437; DPO+MLC reached 95.94 average, variance 0.07, and PAG 0.9697. The raw model scored 59.55 average, variance 13.14, PAG 0.5037; MPO scored 65.50, variance 5.53, PAG 0.6979; SDRRL scored 70.28, variance 0.45, PAG 0.8412. DPO+MLC kept all ten languages above 90% safety.
• The pattern repeats on Gemma-2-9B-it. DPO+MLC achieved 96.83 average safety, variance 0.02, PAG 0.9989, against raw 81.78 / 6.97 / 0.7390, DPO 81.52 / 6.52 / 0.7793, MPO 78.95 / 1.79 / 0.8942, and SDRRL 71.94 / 0.89 / 0.7288.
• Out-of-distribution robustness on MultiJail (lower ASR is better). For Qwen-2.5-7B-Instruct, DPO+MLC attained an in-distribution average ASR of 0.70 and out-of-distribution average of 0.51, compared with DPO 10.54 / 3.62, MPO 4.19 / 3.62, SDRRL 16.88 / 8.38, and raw 17.27 / 8.06. For Gemma-2-9B-it, DPO+MLC reached 0.32 / 0.38 versus DPO 2.73 / 2.79, MPO 3.68 / 2.48, SDRRL 20.19 / 28.89, and raw 4.38 / 4.13. The OOD languages in MultiJail (including Indonesian, Vietnamese, and Thai) were not explicitly seen during training.
• General capability is largely preserved, with a divergence between backbones. On Qwen 2.5-7B, MMLU went from 76.37 (raw) to 76.30 with DPO+MLC, while MMMLU-lite All moved from 55.16 to 52.66. On Gemma-2-9B-it, MMLU moved from 74.34 to 73.58 and MMMLU-lite All improved from 49.25 to 54.61. The paper attributes Qwen's slight multilingual decline and Gemma's improvement to differences in multilingual representation robustness.
• Data efficiency. Building on 0.59 × 10^6 tokens of standard DPO training, MLC uses about 1.8 × 10^6 tokens, against roughly 15 × 10^6 for MPO and ~64 × 10^6 for LDR.
• Scaling behavior (Qwen-2.5 at 1.5B, 3B, 7B). Monolingual DPO's cross-language variance grows with model size (variance 8.76 at 1.5B, 11.69 at 3B, 12.44 at 7B), while DPO+MLC reduces variance to 0.05, 0.18, and 0.70 respectively, with average safety of 94.11, 92.45, and 95.94. On the 7B model the paper reports that DPO+MLC lifts all languages above 90% and reduces variance by over 90% relative to DPO.
• Compatibility across alignment paradigms. Integrating MLC into SFT, DPO, SimPO, and ORPO consistently improves multilingual safety, especially for low-resource languages such as SW and KU, while preserving performance in EN and ZH. Larger gains appear with stronger base paradigms (DPO, SimPO) than weaker ones (SFT, ORPO), which the paper reads as MLC propagating signals from well-optimized anchors.
• Representational geometry changes as predicted. Gram matrices (Z^T Z) averaged over 20 randomly sampled test cases show that DPO+MLC produces uniformly high inter-language similarity, collapsing languages into a shared embedding manifold, while raw, DPO, SDRRL, and MPO show uneven or weak cross-lingual similarity.
• Extraction layer matters. Deeper layers give higher average multilingual safety and PAG, but MMMLU-lite performance is non-monotonic and drops when the final layer is used, while MMLU stays stable across layers — a trade-off between safety alignment and multilingual utility.

Methodology in Plain English

The starting observation is that if the model internally represents the same question the same way regardless of the language it is asked in, its answers should behave consistently too, including on safety.

For each training prompt, the researchers collect its translations into m languages. Each translated prompt is passed through the model, and the hidden state of the last prompt token is taken from a designated transformer layer. A simple linear projection maps that hidden state into a representation vector, which is normalized to unit length. The m vectors are stacked side by side into a matrix.

The goal is to make those vectors collinear — pointing along one shared semantic direction. The paper shows, citing a standard linear algebra result, that a set of vectors is collinear if and only if the matrix they form has rank 1. Maximizing collinearity therefore becomes minimizing the distance between the representation matrix and its best rank-1 approximation. Using the Eckart–Young–Mirsky theorem, the best rank-1 approximation keeps only the top singular component, and the error equals the sum of squares of the remaining singular values. A proposition in the paper states that minimizing this error is equivalent to maximizing the relative dominance of the largest singular value.

To make this trainable, the singular values are treated as unnormalized logits, passed through a temperature-scaled softmax, and the loss pushes the distribution to concentrate on the largest one. The final objective is simply the original alignment loss (SFT, DPO, SimPO, or ORPO) plus a weighting term times this consistency loss, trained in a single update. Only prompt translations are needed; no target-language responses are used.

Why This Matters

Impact on research: The paper reframes multilingual safety alignment as a representation-level consistency problem rather than a per-language data problem, and offers a spectral formulation with stated theoretical grounding. It also reports evidence that monolingual alignment in English weakly bleeds into other languages without reducing cross-lingual disparity — a finding relevant to how the field measures alignment success.

Real-world applications:

• Global content moderation and abuse prevention, where a model must refuse the same harmful request regardless of the language it arrives in.
• Consumer and enterprise chatbots deployed in low-resource languages (for example Bengali, Swahili, Urdu, Pashto, Kurdish) where labeled safety data is scarce.
• Compliance and risk management for organizations that must demonstrate consistent safety behavior across all markets they serve, not just English-speaking ones.
• Low-budget alignment for smaller teams or resource-constrained deployments, given the reported token cost of about 1.8 × 10^6 versus roughly 15 × 10^6 for MPO and ~64 × 10^6 for LDR.

Industry relevance: The method is designed as an auxiliary loss that drops into existing SFT, DPO, SimPO, or ORPO pipelines, which lowers the barrier to adoption. Reported results on Qwen-2.5 and Gemma-2 backbones across 1.5B, 3B, 7B, and 9B scales suggest relevance for both small and mid-sized deployments.

Future Directions

• Layer-aware alignment strategies. The layer-depth study finds a trade-off — deeper layers improve safety and PAG but the final layer can hurt multilingual utility — and the paper explicitly motivates future work on choosing or adapting the extraction layer.
• Other target properties beyond safety. The definition of multilingual consistency in the paper covers properties such as safety and value stance, so the same mechanism could be tested on other alignment attributes.
• Extending and verifying transfer to unseen languages. MultiJail already includes languages not seen in training (Indonesian, Vietnamese, Thai, with KO, IT, JV, TH, VI also evaluated); how far this generalization extends is an open question.
• Resolving the safety-versus-utility divergence. Qwen showed a slight multilingual capability decline while Gemma improved; understanding why and designing constraints that protect general-purpose features remains open.

The available paper content does not report an explicit limitations section or negative-result analysis beyond these points.

Target Audience

Researchers working on LLM safety alignment, multilingual NLP, and representation engineering will find the main content useful, particularly those interested in low-resource language transfer and hidden-state analysis. Practitioners who need multilingual guardrails but lack per-language safety data will benefit from the plug-and-play formulation and the reported token costs. Readers should be comfortable with linear algebra (SVD, rank, Frobenius norm) and with standard alignment objectives such as SFT and DPO to get the most from the methodology and propositions.

Note: the paper carries a warning that it contains example data that may be offensive or harmful.

Authors’ abstract

The widespread deployment of large language models (LLMs) across linguistic communities necessitates reliable multilingual safety alignment. However, recent efforts to extend alignment to other languages often require substantial resources, either through large-scale, high-quality supervision in the target language or through pairwise alignment with high-resource languages, which limits scalability. In this work, we propose a resource-efficient method for improving multilingual safety alignment. We introduce a plug-and-play Multi-Lingual Consistency (MLC) loss that can be integrated into existing monolingual alignment pipelines. By improving collinearity between multilingual representation vectors, our method encourages directional consistency at the multilingual semantic level in a single update. This allows simultaneous alignment across multiple languages using only multilingual prompt variants without requiring additional response-level supervision in low-resource languages. We validate the proposed method across different model architectures and alignment paradigms, and demonstrate its effectiveness in enhancing multilingual safety with limited impact on general model utility. Further evaluation across languages and tasks indicates improved cross-lingual generalization, suggesting the proposed approach as a practical solution for multilingual consistency alignment under limited supervision.