STAR : Sentence Translation Alignment Rate for Document-to-Document Machine Translation

AI summary

Overview

• Research area: Document-level machine translation (DocMT), specifically document-to-document (Doc2Doc) translation with large language models, along with document-level evaluation and preference-based optimization.
• Technical level: Intermediate. The paper assumes familiarity with machine translation metrics (COMET, BLEU), LLM fine-tuning (SFT, CPO, DPO), and sentence alignment, but the core ideas are explained through clear structural analogies.
• Scope: The paper introduces a sentence-level structural fidelity metric (STAR) and a masking-based preference optimization framework (StarPO) to reduce sentence omissions and hallucinations in single-pass document translation.

What This Paper Is About

Large language models can now translate an entire document in one pass, but doing so often breaks the sentence-by-sentence correspondence between source and target: the model drops sentences (omissions) or invents new ones (hallucinations). Standard translation metrics such as COMET mostly measure semantic adequacy and fluency, so they can score such structurally broken outputs highly. This paper proposes STAR, a metric that explicitly measures sentence-level alignment, and StarPO, a training framework that uses STAR to build preference data and applies a sentence-level mask so optimization concentrates on the structurally misaligned parts of a translation.

Key Contributions

1. Identification of structural misalignment as a bottleneck: The authors frame sentence-level structural failure (omissions, hallucinations, and other non-1-to-1 alignments) as a key limitation of single-pass Doc2Doc translation, and show it persists across model scales and even when sentence-boundary constraints are added during generation.
2. STAR metric: A Sentence Translation Alignment Rate that segments source and target documents, aligns sentences into minimal units, categorizes those units as 1-to-1, deletion (1-to-0), insertion (0-to-1), or complex, and reports the fraction of clean 1-to-1 units. A relaxed variant, STAR_relax, counts complex units as positive and penalizes only omissions and hallucinations.
3. StarPO framework: STAR-masked Preference Optimization, which ranks translation candidates by STAR to build chosen/rejected pairs and applies a dynamic sentence-level mask that zeroes out well-aligned (1-to-1) sentences so the contrastive preference loss focuses on structurally problematic segments.
4. Empirical validation across domains and models: Consistent improvements on WMT25 News-Commentary and Guofeng web-novel datasets with LLaMA-3.1-8B-Instruct, Qwen-2.5-7B-Instruct, and Qwen-3-4B-Instruct, including surpassing Tower-plus-9B, GPT-4o, and Deepseek-R1 on average scores.

Main Findings

• Structural errors are pervasive: In the preliminary analysis (Zh⇒En on News-Commentary, scored by Gemini-2.5-Flash), LLaMA-3.1-8B produced 92.59% 1-to-1 alignments, 2.08% 1-to-0, 1.17% 0-to-1, and 4.16% other; GPT-4o produced 92.91% 1-to-1, 2.25% 1-to-0, 2.89% 0-to-1, and 1.95% other. Sentence-boundary-aware Chunk2Chunk systems such as MixSFT (96.78% 1-to-1) and KFMT (95.48%) still show non-1-to-1 rates.
• Best alignment distribution with StarPO: The authors' method on Qwen2.5-7B reaches 98.43% 1-to-1, 0.68% 1-to-0, 0.00% 0-to-1, and 0.89% other, the best 1-to-1 rate in Table 1.
• Quality gains on News-Commentary (dCOMET, wmt22-comet-da): StarPO gives the best average across the five language pairs for every backbone. Averages are 81.28 for LLaMA-3.1-8B-Instruct (+StarPO vs. 80.80 with +CPO, 80.01 with +SFT, 75.07 Base), 81.42 for Qwen2.5-7B-Instruct (vs. 81.16, 80.91, 80.19), and 81.73 for Qwen3-4B-Instruct (vs. 81.46, 81.34, 81.03). Comparison systems average 81.10 (Tower-plus-9B), 80.72 (GPT-4o), and 80.97 (Deepseek-R1).
• LLaMA-3.1 gain over CPO: On LLaMA-3.1, StarPO obtains an average improvement of 0.48 COMET over standard CPO.
• Quality gains on News-Commentary (d-BLEU): Averages under StarPO are 35.59 (LLaMA-3.1-8B-Instruct), 35.47 (Qwen2.5-7B-Instruct), and 35.81 (Qwen3-4B-Instruct), versus 35.12 (Tower-plus-9B), 34.75 (GPT-4o), and 34.41 (Deepseek-R1).
• Literary domain results (Guofeng dCOMET): StarPO improves all backbone model families; for example, LLaMA-3.1 on Zh⇒De rises from 52.31 with the Base model to 72.15 with StarPO, which the authors cite as restoring coherence where the base model fails.
• StarPO beats alternative preference-data ranking signals: With the same data budget on LLaMA-3.1-8B-Instruct Zh⇔En, StarPO scores 81.55 (Zh⇒En) and 80.11 (En⇒Zh), compared with 81.10/79.94 for CPO, 81.01/79.78 for COMET ranking, 80.56/79.73 for COMETKiwi ranking, 76.55/79.71 for BLEU ranking, and 75.94/79.46 for word-level coverage.
• StarPO beats online RL baselines tested: GSPO with STAR is the closest competitor at 80.16/80.00; GRPO variants range from 77.73 to 77.81 on Zh⇒En. The authors note that STAR nevertheless functions as a robust reward signal in RL paradigms.
• Ablations: Training only on preferred responses (SFT on preference data) gives 80.41/80.02; using the relaxed STAR variant gives 80.09/79.59; random sentence-level masking gives 81.18/80.05 and random token-level masking 81.17/80.06 — all below StarPO's 81.55/80.11.
• STAR correlates better with structural quality than alternatives: Spearman correlations with LLM-annotated alignment quality (Gemini-2.5-Pro as annotator, strict / relaxed) are 0.5808/0.5774 for STAR; 0.3804/0.5168 for Align-then-Slide; 0.4193/0.4760 for SEGALE; and much weaker for length-based metrics — Token Count Ratio 0.0218/0.0211, Sentence Count Ratio 0.1218/0.1780, Sentence Count Difference 0.0649/0.1647.
• STAR is robust to component swaps: Replacing SaT with Spacy drops correlation to 0.5644/0.5663 and switching LaBSE to M3 to 0.5104/0.5479.
• Where the gain comes from (Zh⇒En fine-grained COMET): StarPO reduces pathological errors from 3.25% (Base) to 2.09% of segments, and improves the COMET score of complex restructuring segments from 71.87 (Base) to 81.52; 1-to-1 segment COMET rises from 75.78 to 83.33, and overall from 73.15 to 81.55.
• LLM-as-a-judge results (News-Commentary Zh⇒En, Gemini-2.5-Flash): StarPO is best across all dimensions. LLaMA-3.1 fluency/content/cohesion: 3.99/1.40/1.27 (StarPO) vs. 3.97/1.90/1.58 (CPO) and 3.67/2.33/1.95 (Base). Qwen-2.5-7B: 4.45/1.16/0.95. Qwen-3-4B: 4.59/1.17/0.89. Comparison systems: Tower-plus 4.05/1.29/1.14, GPT-4o 4.18/1.26/1.12, Deepseek-R1 4.37/1.37/1.20.
• Threshold and statistics: Preference pairs are kept only when the STAR difference exceeds τ = 0.1; declared significant improvements need at least 85% agreement with human judgment, i.e. ≥ 0.71 COMET for wmt22-comet-da and ≥ 3.35 BLEU for d-BLEU.

Methodology in Plain English

The approach has three connected pieces.

First, measuring structure. Given a source document and its translation, the system splits both into sentences using SaT and aligns them with Bertalign. The resulting minimal alignment units are labeled 1-to-1, deletion (1-to-0), insertion (0-to-1), or complex (everything else). STAR is simply the share of units that are clean 1-to-1. The relaxed version treats complex units as acceptable and only punishes omissions and hallucinations, on the reasoning that merging, splitting, or reordering is often legitimate for fluency in the target language. STAR can also be computed by prompting an LLM judge with the same four steps.

Second, building preference data. For each source document, GPT-4o generates 5 candidate translations at temperature 1.0, and any available reference translation without sentence boundaries is added to the pool. Each candidate is scored with STAR; the highest-scoring candidate becomes the "chosen" example and the lowest becomes the "rejected" one. A pair is kept only if the STAR gap exceeds τ = 0.1, ensuring the supervision signal is meaningful.

Third, training. The model first undergoes supervised fine-tuning on high-quality parallel corpora to create a warm-started policy, then is optimized with a contrastive preference objective (CPO) adapted for structure. In standard CPO, the likelihood of the preferred translation is compared with the rejected one over all tokens. StarPO changes this by multiplying each sentence's log-probability by a mask that is 0 for sentences in clean 1-to-1 alignments and 1 otherwise. Well-aligned sentences therefore stop contributing to the loss, and gradient updates are driven by the misaligned segments — the omissions, hallucinations, and complex restructuring that the method is designed to fix.

Evaluation uses document-level COMET (wmt22-comet-da), where unaligned omissions and hallucinations are assigned a score of 0 while complex mappings keep their computed COMET scores, and d-BLEU, which treats the whole document as one continuous string to avoid sentence-boundary artifacts.

Why This Matters

• Impact on research: The paper argues that structural misalignment is largely invisible to standard training objectives and metrics, so Doc2Doc systems can score well while silently dropping or inventing content. STAR provides a diagnostic metric, and the strong correlation with LLM-annotated alignment quality (0.5808 strict, 0.5774 relaxed) positions it as a more reliable structural signal than length-based heuristics or reusing the intermediate alignments of existing evaluation methods such as Align-then-Slide and SEGALE.
• Practical implications for translation workflows: Because STAR and StarPO target omissions and hallucinations, they are relevant wherever missing or fabricated content is costly.
  • Localization and enterprise content: Product documentation, legal contracts, and manuals where a dropped sentence changes meaning or creates liability.
  • News and media translation: Wire services republishing translated journalism, where hallucinated sentences are a credibility risk.
  • Literary and web-novel translation: The Guofeng experiments show gains on stylized text where merging and splitting are common, and where base models can lose coherence entirely (LLaMA-3.1 Zh⇒De at 52.31 COMET).
  • Subtitle and long-form media translation: Content where global coherence across a whole document matters and sentence-by-sentence translation loses discourse context.
• Industry relevance: The headline claim is that smaller, cheaper models can match or beat massive proprietary systems. StarPO averages exceed GPT-4o on both dCOMET and d-BLEU across the News-Commentary pairs, and the framework replaces complex agentic or multi-turn pipelines with a single-pass model plus offline preference optimization. The authors also describe token efficiency as a benefit, and present an alignment-score-versus-token-consumption comparison (plotted on a logarithmic x-axis) against Doc2Sent systems (w=3 sliding window, Source-primed, DelTA), Doc2Doc systems (KFMT, Mix-level SFT, DocRefine), and large baselines (GPT-4o, DeepSeek-R1); the specific token counts are not reported in the provided text.

Future Directions

• Scaling to larger models and lower-resource languages: The authors explicitly state that validation is limited to compact models of 4B to 9B parameters and high-to-medium resource languages, and that 70B+ architectures and low-resource languages remain untested. This is stated as a critical direction for future research.
• Removing dependence on proprietary APIs: Candidate translation diversity is currently produced with GPT-4o. Replacing this step with open-source alternatives would enable a fully offline-deployable, end-to-end open-source pipeline.
• Relaxing the 1-to-1 constraint without losing the benefit: The authors acknowledge that enforcing 1-to-1 alignment imposes structural rigidity that could theoretically discourage valid complex mappings in stylized texts, even though their empirical results suggest minimal harm. Finding a way to permit legitimate restructuring while still suppressing omissions and hallucinations is open.
• Reducing the alignment prerequisite: The paper notes that establishing sentence-level alignment is an unavoidable prerequisite for computing fine-grained quality metrics such as d-COMET in single-pass Doc2Doc scenarios, which leaves room for methods that assess structural fidelity without a separate alignment step.

Target Audience

• Machine translation researchers working on document-level translation, evaluation metrics, and structural fidelity.
• Preference-optimization and RLHF practitioners interested in how data ranking signals and loss masking choices affect downstream behavior, particularly the finding that semantic-aware masking beats random masking and that a relaxed ranking metric degrades results.
• Applied NLP engineers and localization teams who need to deploy document translation while controlling hallucinations and omissions, especially those constrained to small open models.
• Students and newcomers to DocMT who want a clear framing of the Sent2Sent, Chunk2Chunk, and Doc2Doc paradigms and their trade-offs; the core metric is defined in four explicit steps and is accessible without deep background.

Authors’ abstract

Large Language Models (LLMs) have enabled a shift from sentence-level to document-to-document (Doc2Doc) machine translation, promising improved global coherence. However, document-to-document generation in a single pass frequently suffers from structural misalignment, manifesting as sentence omissions or hallucinations that violate the core requirement of source-target correspondence. To address this, we introduce Sentence Translation Alignment Rate (STAR), an auxiliary metric that explicitly quantifies sentence-level structural fidelity. Building on this, we propose STAR-masked Preference Optimization (StarPO), a framework that ranks document-level hypotheses by structural quality and utilizes a dynamic alignment mask to focus optimization on misaligned segments. Experimental results across news and literary domains demonstrate that StarPO significantly enhances translation quality and structural integrity. Notably, StarPO allows compact models to surpass the performance of massive proprietary systems like GPT-4o while maintaining superior token efficiency.