It Takes Two: A Dual Stage Approach for Terminology-Aware Translation

AI summary

Overview

Research area: Terminology-constrained machine translation (NLP), specifically the WMT 2025 Terminology Shared Task.

Technical level: Intermediate. The paper assumes familiarity with neural machine translation, fine-tuning, terminology constraints, and automatic evaluation metrics like BLEU and chrF2++, but the architecture itself is conceptually straightforward.

Scope: The paper describes DuTerm, a two-stage system pairing a fine-tuned, terminology-tagged NLLB-200 3.3B model with GPT-4o post-editing, evaluated on English-to-German, English-to-Spanish, and English-to-Russian.

What This Paper Is About

Translating specialized vocabulary consistently and accurately is hard for machine translation systems, especially in fields like law, medicine, and engineering where precision matters. Two existing families of solutions exist: inference-time methods that force terminology into the decoder (which can produce awkward, ungrammatical output) and training-time methods that teach a model terminology through tagged data (which offer no guarantee that constraints are respected at inference). DuTerm combines both worlds: a terminology-aware NMT model trained on synthetic tagged data, followed by a prompted LLM that post-edits the NMT output and enforces terminology.

Key Contributions

1. A two-stage architecture (DuTerm) that chains a terminology-aware fine-tuned NMT model with a prompt-based LLM post-editor, directly targeting the WMT 2025 Terminology Shared Task for English-to-German, English-to-Spanish, and English-to-Russian.
2. A synthetic data pipeline using GPT-4o to generate tagged, context-rich parallel sentences in two modes (single-term, and multi-term with 2–3 randomly selected term pairs per sentence), followed by re-tagging, quality filtering with COMET_QE, and source-side deduplication.
3. A multilingual adaptation recipe for NLLB-200 3.3B that extends the model's vocabulary with terminology markup tokens so that [TERM] boundaries are not fragmented by subword tokenization, trained on combined filtered data from all three target languages.
4. An empirical comparison of three terminology handling strategies (noterm, proper, random) using BLEU, chrF2++, and terminology success rates, presented as evidence that flexible LLM-driven terminology handling compares favorably with rigid external constraint enforcement.

Main Findings

• Proper (strict) enforcement produces the highest BLEU and chrF2++ in every language pair. English-to-German reaches 48.06 BLEU / 70.74 chrF2++; English-to-Spanish reaches 58.51 BLEU / 76.08 chrF2++; English-to-Russian reaches 35.80 BLEU / 63.57 chrF2++. Proper-term success rates are near-perfect at 0.98 (DE), 0.99 (ES), and 0.98 (RU).
• Unconstrained translation (noterm) underperforms across the board. BLEU/chrF2++ are 38.24 / 62.61 for German, 45.98 / 67.05 for Spanish, and 27.88 / 55.29 for Russian, with proper-term success rates of 0.43, 0.47, and 0.39 respectively.
• Random terminology enforcement lands in the middle on quality but near-perfect on its own constraint. Random success rates are 0.99 (DE), 0.98 (ES), and 0.99 (RU), with intermediate BLEU/chrF2++ of 43.77 / 67.22 (DE), 53.28 / 72.05 (ES), and 32.25 / 59.85 (RU). The paper reads this as evidence that a model can be forced to use arbitrary terminology, but doing so compromises contextual appropriateness.
• Spanish scores highest and Russian shows the widest proper-versus-noterm gap. The paper attributes the Spanish result to its structural similarity to English and the Russian gap to the difficulty of controlling terminology in morphology-rich languages.
• The paper's stated conclusion is that flexible, context-driven terminology handling by the LLM often yields higher translation quality than strict terminology enforcement, while the results table simultaneously shows strict enforcement topping BLEU and chrF2++. The paper reports both without fully reconciling them, noting that strict enforcement "can occasionally reduce flexibility" while unconstrained approaches risk terminology inconsistency.
• The paper's intuition for why post-editing helps: NMT excels at strict word-level translation but struggles with context-dependent nuance. Starting from a partially correct NMT sentence lets the LLM focus on resolving ambiguities, adjusting word order, and refining context, rather than handling term accuracy and contextual fluency simultaneously.

Methodology in Plain English

The team first built bilingual terminology dictionaries by parsing the WMT 2025 development files for English-to-German, Spanish, and Russian. Each direction yields dictionaries that typically exceed 1,000 unique pairs; the LLM is also used to generate additional terms similar to the provided ones. Terms are tracked with repetition_ids.

Next they used GPT-4o to write synthetic parallel sentences that naturally contain those terms, wrapping each term in [TERM] and [/TERM] tags on both source and target sides. Two generation modes are used: single-term (exactly one term instance per sentence) and multi-term (2–3 term pairs appearing together, to teach co-occurrence and disambiguation). Generation uses temperature sampling between 0.3 and 0.7 with concurrent generation and strict parsing.

A re-tagging pass standardizes annotation: longest-first matching avoids partial shadowing, detection is case-insensitive while preserving original case, and inverse mapping keeps target-side tagging symmetric. Each pair is then scored with COMET_QE. Duplicates are removed on the source side and only high-confidence items are kept using a conservative threshold of 0.85–0.9 depending on language, typically retaining 60–70% of outputs, for roughly 10k–15k pairs per language direction.

The base translation model is NLLB-200 3.3B. Its vocabulary is extended with terminology markup tokens so the markup is treated atomically rather than split into subwords. Training uses parameter-efficient fine-tuning and combines the filtered datasets from all three target languages into a single unified multilingual adaptation that benefits from cross-lingual transfer.

The second stage is GPT-4o used as a post-editor. Prompts present the source, the current translation, and explicit source-to-target term mappings, instructing the model to preserve meaning, apply exact target terms, maintain tags, and improve readability. Three behaviors are specified: dynamic resolution (per-input selection of proper, random, or no-term constraints from reference terminology databases with whitespace-normalized matching), mode-adaptive behavior (enforce constraints when they exist, otherwise perform quality-only edits while staying sensitive to technical terms), and explicit constraint satisfaction rules in the prompt. The LLM runs at a low temperature of 0.3 for deterministic edits. Each hypothesis is validated for format, tag integrity, and constraint satisfaction by a pre-existing parser before acceptance, and output files are checked for filename schemas, presence of all terminology modes per language pair, and JSONL structure. Quality is measured with COMET_QE after tag stripping, and terminology preservation is computed via exact-match checks on required terms.

Why This Matters

The paper speaks directly to a practical deployment question in translation: should terminology be imposed as a hard constraint, or should a strong language model be trusted to handle it with guidance? Its results sharpen that trade-off and provide a reusable data-generation and post-editing recipe for anyone building terminology-constrained systems.

Real-world applications:

• Legal and regulatory translation, where fixed terms of art must map to approved equivalents.
• Medical and clinical documentation, where terminology consistency affects safety and downstream interpretation.
• Engineering and technical manuals, where component names and part designations must remain stable across documents.
• Business and financial localization, the domain the paper's prompts explicitly target (IT and financial content), where brand and product terms must be rendered consistently.

Industry relevance: The approach uses off-the-shelf components — an open multilingual NMT model and a commercial LLM API — plus synthetic data rather than large human-annotated corpora, which lowers the barrier to building terminology-aware pipelines. The finding that a well-structured NMT baseline plus LLM post-editing beats translating from scratch is directly actionable for translation vendors deciding where to insert an LLM into an existing MT pipeline. The conclusion also notes the potential of prompt-based LLM systems for technical and business translation tasks.

Future Directions

• Adaptive learning mechanisms that integrate terminology dynamically rather than through static prompts, to improve robustness across domains and languages.
• End-to-end or memory-augmented architectures that maintain consistency across sentences and documents, since the current system operates at the sentence level and cannot enforce document-level terminology consistency.
• Broader evaluation across other language models and diverse, domain-specific corpora. The paper's limitations note that evaluation was conducted solely on GPT-4o and focused on technical and business domains, leaving fields like medical or legal translation untested.
• Hybrid strategies combining prompt guidance with fine-tuning or reinforcement learning, plus user-driven interaction for terminology control, to improve usability and accuracy.

The paper also flags several open concerns: prompt dependence and its limited generalization across domains, languages, and model architectures; the sequential (rather than adaptive) processing of terminology matching and translation refinement; and the fact that COMET_QE, BLEU, and chrF++ may not fully reflect terminological precision and contextual appropriateness, suggesting a need for complementary human judgment. The paper does not report human evaluation results.

Target Audience

Researchers and engineers working on machine translation, terminology management, or domain adaptation will get the most from this paper, particularly those participating in or following WMT shared tasks. It is also useful for localization practitioners evaluating whether to insert an LLM post-editing stage into an existing NMT pipeline, and for readers interested in the practical division of labor between constrained decoding, training-time tagging, and prompted post-editing. Readers need comfort with standard MT evaluation metrics and fine-tuning concepts; the architecture itself is easy to follow.

Authors’ abstract

This paper introduces DuTerm, a novel two-stage architecture for terminology-constrained machine translation. Our system combines a terminology-aware NMT model, adapted via fine-tuning on large-scale synthetic data, with a prompt-based LLM for post-editing. The LLM stage refines NMT output and enforces terminology adherence. We evaluate DuTerm on English-to German, English-to-Spanish, and English-to-Russian with the WMT 2025 Terminology Shared Task corpus. We demonstrate that flexible, context-driven terminology handling by the LLM consistently yields higher quality translations than strict constraint enforcement. Our results highlight a critical trade-off, revealing that an LLM's work best for high-quality translation as context-driven mutators rather than generators.