Reasoning Before Translation: Enhancing Legal Machine Translation with Structured Reasoning

AI summary

Overview

Research area: Natural language processing, specifically neural machine translation (NMT) for the legal domain, combining supervised fine-tuning (SFT), reinforcement learning (RL), and reasoning-capable language models.

Technical level: Intermediate.

Scope: A comparative study of three ways to inject reasoning into Swiss legal translation — prompting frontier reasoning models, SFT on intermediate reasoning steps, and GRPO-based reinforcement learning with verifiable rewards — across small models of 4B to 12B parameters.

What This Paper Is About

Legal translation is hard because of specialized terminology, complex sentence structure, and the need for exact preservation of legal meaning; Switzerland is an especially demanding testbed because German, French, Italian, and Romansh are official languages. The paper asks whether reasoning — either elicited by prompting, taught through supervised fine-tuning, or optimized through reinforcement learning — can close the quality gap between small, deployable language models and large frontier models. The authors systematically compare these strategies on Swiss legal text.

Key Contributions

1. A systematic comparison of three ways to integrate reasoning into translation across model families: prompting large reasoning models, fine-tuning small language models on intermediate reasoning steps, and applying RL to optimize translation.
2. Release of a new dataset of 40,000 legal translation pairs enriched with automatically generated intermediate reasoning steps produced using DeepSeek-R1.
3. A GRPO training recipe with a simple reward formulation tailored to legal translation, intended as a blueprint for domain-specific reinforcement learning in NMT.
4. Public release of code and models at https://github.com/aixiuxiuxiu/Legal-MT-SFT-RL.

Main Findings

• RL beats SFT consistently: Reinforcement learning improved translation quality on all three base models (Qwen3.5 4B, Qwen3.5 9B, Gemma 3 12B) and outperformed both SFT variants on every model.
• Reasoning SFT hurts quality: Fine-tuning on intermediate reasoning steps degraded translation relative to Simple SFT, and the damage was largest on the smallest model, Qwen3.5 4B (COMET 69.88 for Reasoning SFT versus 76.47 for Simple SFT).
• Simple SFT is marginally worse than doing nothing: Simple SFT performed slightly below the unaltered base models (for example, Qwen3.5 4B base COMET 77.05 versus 76.47 for Simple SFT), so using the pretrained model as-is was more beneficial than SFT.
• Best small model is Gemma 3 12B plus RL: Among the evaluated smaller-scale models, GRPO-trained Gemma 3 12B achieved the best results (ChrF 57.32, COMET 83.08, METEOR 54.92, MetricX 3.32), performing just below the frontier model o4-mini.
• Frontier models still lead: On prompting, OpenAI o3 achieved the highest scores across all metrics (ChrF 64.87, COMET 85.92, METEOR 64.82, MetricX 2.54) but was also the most expensive at $8.00 per 1M output tokens, whereas Mistral-Small-2503 was roughly ten times cheaper at $0.15 while delivering comparable results.
• The gap is narrowing: Figure 3 reports commercial frontier models at COMET 84.21 to 85.94 versus RL-enhanced small models at 79.47 to 83.08, which the authors describe as nearly overlapping.
• Diminishing returns with scale: RL's relative gain shrank as models grew — Qwen3.5 4B improved from 77.05 to 79.47 (+2.47 COMET), while Gemma 3 12B improved only from 82.49 to 83.08 (+0.59).
• Best-performing open-source models use more than 100x fewer parameters than frontier models while achieving competitive translation quality.
• Metric agreement: The ranking of methods was consistent across ChrF, COMET, METEOR, and MetricX, even though the metrics capture different aspects of quality.
• Human evaluation favors COMET: On 240 sampled translation pairs scored by a legal expert on a 1-to-10 scale, COMET had the strongest Pearson correlation with expert judgment (r = 0.24, p = 0.00), followed by METEOR (0.07), ChrF (0.05), and MetricX (-0.13).
• Language-pair patterns: All models performed best on fr to it pairs, likely due to the close relationship between the two Romance languages. o4-mini was notably strong translating into English, while Mistral-Small-2503 performed better into French or Italian. For the small models, the strongest results were on de to en, and fr to it showed the largest gap versus frontier models.

Methodology in Plain English

The authors start from SwiLTra-Bench, described as the largest publicly available Swiss legal translation dataset, using its SwissLawTranslations (CH-Law-Trans) subset: roughly 631,000 aligned training pairs and 22,700 sentence pairs for evaluation.

To build reasoning data, they take the first 40,000 sentence pairs from the training split and prompt DeepSeek-R1 to produce both reasoning steps and a final translation. They score the outputs with ChrF and keep only those above the median value of 64.19, retaining about half the data — a final dataset of 19,979 sentence pairs, split 90 percent for training and 10 percent for validation. The paper's Table 4 reports a total of 18,981 pairs distributed across language pairs (fr to it 8,152; de to fr 5,384; de to it 4,252; de to en 65; fr to en 65; it to en 63).

For evaluation, they use the 22,700-pair test split, exclude Romansh because the translation models do not support it, leaving 18,100 sentence pairs.

Three experiments follow. Experiment I prompts frontier models (DeepSeek-R1, o4-mini, o4, o3, o3-mini, plus Mistral-Small) on Azure AI Foundry using a single direct translation prompt. Experiment II fine-tunes Qwen3.5 4B, Qwen3.5 9B, and Gemma 3 12B in two ways: Simple SFT on translation pairs only, and Reasoning SFT on the reasoning steps plus the ground-truth translation, using an Alpaca-style instruction format. Fine-tuning used Hugging Face transformers and unsloth with 4-bit quantization, 8-bit AdamW, LoRA with rank 16 and alpha 32, on two NVIDIA A40 48GB GPUs, with sequence packing, weight decay 0.01, batch size between 40 and 64, 10 epochs, early stopping with patience 3, a cosine learning rate schedule with warmup ratio 0.1, and a base learning rate of 1e-4 tuned within [2e-5, 1e-4]. Maximum sequence length was 512 tokens for Simple SFT (about 99 percent of sentences) and 1024 for Reasoning SFT (over 95 percent of the training set). Generation used vLLM at temperature 0, which was chosen after testing the range [0, 0.7]. Generation over the full evaluation set takes around 30 minutes for Simple SFT models and 3 to 6 hours for Reasoning SFT models.

Experiment III applies GRPO with two rule-based rewards. The format reward gives 0.5 for the presence of each of the <think></think> and <translation></translation> tags, plus an additional 0.5 if they occur in the correct order. The translation quality reward is the ChrF score of the extracted translation against the ground truth, used directly because its values lie in [0, 1]. A group size of G = 4 was used, with only a single epoch instead of the 10 used for SFT, and the learning rate had to be reduced to 2e-5 because the higher rate caused training instabilities. Evaluation uses ChrF, METEOR, COMET, and MetricX; GEMBA-MQM was not prioritized because its GPT-based evaluation would cost an estimated $3,000 or more.

Why This Matters

The paper provides evidence that reinforcement learning with verifiable rewards, not supervised imitation of reasoning traces, is what actually improves domain-specific translation for small models. It also offers a negative result with practical value: teaching a model to imitate another model's reasoning steps made translations worse, suggesting that shallow or formulaic reasoning traces do not add genuine capability.

Real-world applications:

• Translating Swiss federal and cantonal legal texts between German, French, and Italian, where accuracy and terminological consistency are legally consequential.
• Deploying on-premise or privacy-constrained translation systems for courts, public administration, and government institutions where sending documents to proprietary APIs is not acceptable.
• Cost-sensitive institutional translation pipelines, since the RL-enhanced small models are expected to cost only a fraction of frontier models that comprise several hundred billion parameters.
• Building domain-specific translation assistants for legal professionals, where the models are open-source and reproducible.

Industry relevance: The work speaks directly to organizations that need high-quality legal translation but face deployment cost, data privacy, or on-premise inference constraints that rule out large proprietary models. It also gives the machine translation community a reusable RL recipe and an open dataset of reasoning-enriched legal translation pairs.

Future Directions

• Improving reinforcement learning methods for legal machine translation, including more effective reward designs beyond the simple rule-based format and ChrF rewards used here.
• Investigating more efficient training strategies and optimizing the trade-off between training and inference cost and translation quality.
• Exploring models of varying sizes and architectures, since the paper fine-tunes only a handful of commonly used models.
• Understanding the diminishing returns of RL as model size increases, which the authors state they plan to investigate further.
• Examining whether more complex or adaptive reward functions, rather than rule-based rewards, could yield additional gains.

Target Audience

Researchers and practitioners working on machine translation, domain adaptation, and reinforcement learning for language models. It is also relevant to computational linguists and legal translation specialists interested in Swiss multilingual legal text, and to engineers at institutions or companies who need to choose between frontier models and fine-tuned open-source models under cost, privacy, and deployment constraints.

Authors’ abstract

Neural machine translation (NMT) in the legal domain is a linguistically and conceptually demanding task, primarily due to the complexity of legal language and the high level of precision it requires. The recent emergence of reasoning-capable language models opens new possibilities for tackling such challenges. They add to a set of other previously proposed techniques to enhance the translation quality, which includes supervised fine-tuning and reinforcement learning. In this work, we perform a comparison between these various approaches. More particularly, we evaluate small language models such as Qwen3.5 4B, Qwen3.5 9B, and Gemma 3 12B enhanced with various re-training paradigms and compare their performances against frontier reasoning models. We focus on the Swiss legal system, which -- with its unique multilingual statutes -- offers a particularly challenging testbed for reasoning-augmented models. Our results show that the quality of small ``base'' models can be greatly enhanced, and that reinforcement learning with verifiable rewards can be applied to NMT in the legal domain and surpasses the translation quality of supervised fine-tuning. The performance of enhanced small models is close to the one of state-of-the-art reasoning models yet remains inferior. We also note that re-training paradigms yield diminishing returns as model size increase. The code and models are publicly available at https://github.com/aixiuxiuxiu/Legal-MT-SFT-RL.