Cross-Lingual Alignment for Decoder-Only Models using MoE Routers

AI summary

Overview

• Research area: Natural Language Processing — multilingual representation learning and cross-lingual transfer in large language models, specifically mixture-of-experts (MoE) architectures.
• Technical level: Intermediate. The method itself is conceptually simple (an auxiliary KL-divergence loss), but readers benefit from familiarity with MoE routing, contrastive learning, and continual pre-training.
• Scope: The paper proposes and evaluates an auxiliary training loss that aligns the mean-pooled MoE router distributions of translated sentence pairs in decoder-only LLMs, testing it across four MoE models and seven languages.

What This Paper Is About

Cross-lingual contrastive learning has long been standard for multilingual encoder models because an encoder produces a natural sequence-level embedding that can be pulled together for translated pairs. Decoder-only LLMs have no such sequence representation, and differing tokenization across languages makes token-level matching between translations difficult, so the large supply of parallel (translated) text goes largely unused for explicit alignment. This paper asks whether the outputs of MoE routers — mean-pooled across a sequence — can serve as an effective alignment target instead of hidden states, and whether encouraging routing agreement across languages improves downstream multilingual performance.

Key Contributions

1. A new alignment target for decoder-only models. The authors propose using mean-pooled MoE routing weights (probability distributions over experts) as the basis for cross-lingual supervision, arguing these pool more reliably over many tokens than high-dimensional hidden states.
2. A concrete loss and training procedure. The cross-lingual routing loss L_XLR is the average KL-divergence between mean-pooled routing distributions of English and target-language sequences across a fixed set of middle layers M, added to the standard language-modeling loss as L = L_LM + α · L_XLR. Gradients are backpropagated only through the target-language sequence.
3. Efficient implementation. A shared packed forward pass over source and target sequences without padding, with variable-length attention preserving sequence boundaries and source tokens exiting early after the last alignment layer, resulting in only a small computational overhead relative to single-language CPT.
4. Controlled empirical study. Experiments on four open-source MoEs (Qwen3-30B-A3B, GPT-OSS-20B, Granite-4.0-H-Tiny, Marco-Nano) and seven languages (Vietnamese, Sinhala, Hungarian, Telugu, Kannada, Thai, Kyrgyz), with comparisons against a baseline, router-only training, and a mean-pooled hidden-state alignment variant.

Main Findings

• Routing alignment increases most with the auxiliary loss. For all models and languages, training with the auxiliary loss produced the biggest increase in routing alignment (i.e., the largest decrease in routing divergence measured by mean entropy-normalized JS-divergence). Baseline CPT led to only minor changes in routing relative to the original checkpoint; for the post-trained Qwen3-30B-A3B, vanilla CPT did not alter routing behavior at all in the reported figure.
• Hidden representations also become more similar. Using the SoftCKA cross-lingual alignment metric on Qwen3-30B-A3B, the routing loss made hidden states more similar to English across Telugu, Kyrgyz, Kannada, and Thai — even though the loss targets router outputs.
• Consistent downstream gains. The routing loss improved over the baseline for 13 of the 14 model–language pairs and tied for the remaining pair (Marco-Nano on Vietnamese). Positive gains ranged up to 2.1 average points, with a mean improvement of 0.9 points across all pairs.
• Example model-level results. Qwen3-30B-A3B: Telugu 45.4 to 46.3, Kyrgyz 43.5 to 45.6, Kannada 49.6 to 51.4, Thai 44.9 to 45.4. GPT-OSS-20B: Telugu 46.5 to 47.0, Kyrgyz 42.0 to 43.8, Kannada 55.5 to 56.4, Thai 39.9 to 40.8. Granite-4.0-H-Tiny: Vietnamese 36.5 to 36.8, Sinhala 32.8 to 32.9, Hungarian 38.1 to 38.8. Marco-Nano: Vietnamese 45.6 to 45.6, Sinhala 26.1 to 26.8, Hungarian 37.4 to 38.1.
• Routing beats hidden-state alignment in the tested ablation. On Marco-Nano, hidden-state alignment moved Sinhala from 26.1 to 26.3 (below the 26.8 from routing alignment) and decreased Hungarian from 37.4 to 36.9, where routing alignment increased it to 38.1. Hidden-state alignment used α = 5000 and routing alignment used α = 10 in both languages.
• Router-only training is not a reliable substitute. Updating fewer than <0.1% of parameters occasionally matches or exceeds the LM-only baseline but varies substantially across settings, with similarly noisy changes to internal alignment metrics. The authors conclude the gains from the routing loss arise primarily from changes to the representations supplied to the routers rather than from updates to the routing function alone.
• Baseline gains are small overall. The baseline yields only modest improvement over the original checkpoint, which the authors attribute to performing a short continual pre-training stage on already post-trained models.

Methodology in Plain English

The researchers take an already-released MoE language model and continue training it on 200,000 English–target-language parallel examples per language (plus 1,000 held-out examples for validation), drawn from curated sources and supplemented with OPUS when fewer curated examples exist.

For each training pair, they run the English and target-language sentences through the model and record what the routers do at each layer — the probability distribution over experts that the router assigns for each token. They average these distributions across all tokens in a sequence, producing one vector per sequence per layer. The auxiliary loss simply measures how different the English average is from the target-language average at selected middle layers, using KL-divergence, and averages that difference across those layers. This is added to the normal next-token-prediction loss, scaled by a hyperparameter α. Only the target-language sequence receives gradients; English is used solely to produce the comparison distributions.

Middle layers were chosen per model using published cross-lingual routing divergence curves rather than exhaustive search — for Qwen3-30B-A3B layers 7–34, GPT-OSS-20B layers 4–17, Marco-Nano layers 7–19, and Granite-4.0-H-Tiny layers 10–35. To save compute, source and target sequences are packed into a single forward pass without padding, and source tokens stop being processed after the last alignment layer.

The trained checkpoints are evaluated with lm-eval-harness on FLORES (English→target generation), Belebele (understanding), the medicine subset of Global-MMLU, MMLU-ProX, MultiLoKo and INCLUDE (local knowledge), Global-PIQA (physical reasoning), MGSM and PolyMath (math). Since language coverage differs across benchmarks, each language's average covers between four tasks (Kyrgyz) and nine (Vietnamese).

Why This Matters

• Impact on research: The paper offers causal evidence that cross-lingual representation alignment improves multilingual performance in modern LLMs, and it introduces MoE routing as an accessible interface for that alignment — one that does not require token-level correspondence between translations, which decoder-only architectures do not provide. It also links the alignment literature from encoder models to the MoE architecture now common in frontier LLMs.
• Real-world applications:
  • Improving LLM quality for languages that are underrepresented in pretraining, such as Sinhala, Kyrgyz, and Kannada.
  • Strengthening cross-lingual knowledge transfer, so capabilities learned in a high-resource language carry over to lower-resource ones.
  • Supporting parameter-efficient adaptation strategies, since router-only training updates fewer than <0.1% of parameters and occasionally matches or exceeds the LM-only baseline.
  • Informing multilingual curriculum design by showing alignment can be applied at any training stage.
• Industry relevance: MoE models are a prominent architecture because they scale parameters at relatively fixed inference cost, and the method adds only a small computational overhead over standard continual pre-training thanks to the packed shared forward pass. The code is released publicly, and the work was supported by the Amazon AI PhD Fellowship.

Future Directions

• Apply alignment during pretraining rather than post-training. The authors believe the method would be far more effective during pretraining, where it could compound across pre-, mid-, and post-training stages; they note prior work suggesting MoE routing dynamics are established quite early in pretraining.
• Address the dependence on parallel data. The method relies entirely on parallel corpora, which are predominantly sentence-level, cover a narrow data distribution, and are absent from much pretraining, SFT, and RL data. The authors frame MoE alignment as one component of a broader training curriculum interleaved with complementary data distributions.
• Explore load balancing and negative pairs. No negative pairs are used, on the reasoning that MoE models are always trained with a load-balancing auxiliary loss that prevents collapse; the interaction between the alignment loss and load balancing at large scale is left to future work.
• Test whether alignment helps transfer-dependent tasks more. The authors hypothesize alignment may benefit tasks relying less on language-specific information (reasoning, math, knowledge), but state that their results do not establish this convincingly over translation and reading comprehension. They also raise cross-modal alignment in multimodal MoE models as a possible extension, motivated by reported modality-dependent routing discrepancies.

Target Audience

This paper is most useful to NLP researchers and engineers working on multilingual LLM training, MoE architectures, or cross-lingual transfer, as well as practitioners who need to adapt post-trained models to lower-resource languages with limited compute. Readers already familiar with contrastive learning and mixture-of-experts gating will get the most from the methodology, though the core idea — aligning averaged router distributions across translated pairs — is explained clearly enough for a motivated newcomer.

Authors’ abstract

Cross-lingual contrastive learning has been a core component of multilingual encoder training, but the ability to explicitly align representations is not possible in decoder-only LLMs because of varying multilingual tokenization. However, a growing amount of research suggests that even in LLMs, higher cross-lingual representational alignment leads to improved cross-lingual transfer. In this paper, we propose a novel approach to reimagine cross-lingual contrastive learning given the architectural constraints of modern LLMs. Rather than applying an auxiliary alignment loss on hidden states, we propose using the outputs of the mixture-of-experts (MoE) routers as the target for alignment. Router outputs lend themselves better to pooling over many tokens, enabling more reliable cross-lingual comparisons at the sequence-level. Controlled continual pre-training experiments on four open-source MoEs show that incorporating this routing loss also aligns the underlying hidden representations across languages. Most importantly, this loss improves multilingual performance on our diverse evaluation suite, demonstrating the potential of cross-lingual MoE router alignment.