HexaBee-4B: Six-Language Translation with Multilingual Answering and Bidirectional Supervision

AI summary

Overview

• Research area: Natural Language Processing, specifically multilingual machine translation and parameter-efficient adaptation of instruction-tuned large language models.
• Technical level: Intermediate. The paper assumes familiarity with supervised fine-tuning, LoRA adapters, continued pretraining, and translation metrics such as BLEU, chrF++, TER, and COMET, but the core idea is explained in accessible terms.
• Scope: The paper describes a three-stage adaptation of Gemma 3 4B IT into a six-language translator called HexaBee-4B, and evaluates it against public Gemma 3 4B IT and Gemma 3 12B IT across all 30 translation directions among Arabic, Chinese, English, Hebrew, Persian, and Spanish.

What This Paper Is About

A general multilingual model can recognize the same information across several languages, but that is not the same as translating: a translator must preserve the meaning of an input while producing text in a specifically requested target language. This paper asks how to organize training data so that answering a question in a chosen language and translating an answer into that language reinforce each other, using the shared multilingual answer as the connecting point. The goal is a compact 4B-parameter translator that outperforms both its own public base model and a larger 12B public model on a synthetic six-language benchmark.

Key Contributions

1. A three-stage adaptation recipe for turning Gemma 3 4B IT into a translator: Wikipedia continued pretraining, joint multilingual answering plus answer-text translation, and bidirectional translation specialization, with two adapter merges in the sequence.
2. An explicit account of how answering and translation share supervision. A single answer text in a given language can serve as the target for both a question-answering task (question in one language, answer in another) and a translation task (answer text in one language rendered into another), and the paper formalizes this with a QA mapping and an MT mapping over the same answer.
3. A fully specified and measured training corpus. Phase 2 comprises 30,153,404 records: 19,458,332 answering records (5,759,806 of them cross-language, or 29.6% of QA supervision) and 10,695,072 answer-text translations, with answering at 64.5% and translation at 35.5% of the mixture. Phase 3 contributes 658,274 bidirectional translation records derived from 329,137 aligned pairs.
4. A direction-complete evaluation covering all 30 directed pairs among the six languages on 36,572 synthetic test examples, using five automatic metrics plus a matched model-judged subset of 300 items, compared against public Gemma 3 4B IT and Gemma 3 12B IT.

Main Findings

• Aggregate advantage on every automatic metric. Direction-macro chrF++ reaches 73.75 for HexaBee-4B, versus 57.11 for Gemma 3 4B IT and 66.83 for Gemma 3 12B IT — a gain of 16.64 points over the same-size base and 6.92 points over the larger comparator. BLEU rises from 40.70 to 62.03, chrF from 58.91 to 75.08, TER falls from 51.40 to 30.05, and COMET rises from 85.88 to 89.86 (with the 12B comparator at BLEU 50.54, chrF 68.70, TER 38.89, COMET 88.20).
• Gains in every direction. The chrF++ improvement is positive for all 30 directions relative to both baselines. Measured against the 4B baseline, the largest directional gain is Chinese-to-Persian (+27.09) and the smallest is English-to-Spanish (+6.64).
• Model-judged results follow the same ranking. Reference-assisted holistic judge means on the matched 300-item subset are 92.92 for HexaBee-4B, 80.32 for 4B IT, and 91.73 for 12B IT; reference-free means are 94.10, 81.43, and 91.97. Paired differences are +12.60 with a 95% interval of [9.96, 15.31] against 4B IT (reference-assisted) and +1.19 with [0.34, 2.08] against 12B IT. The contest with the larger model is markedly closer than with the 4B base.
• A specific metric exception. For Chinese-to-Hebrew, HexaBee-4B has TER 90.09, which is worse than 88.40 for 4B IT and 78.13 for 12B IT, despite higher BLEU and chrF++. TER improves in the other 29 directions. The authors use this to argue against inferring uniform improvement from a single aggregate.
• Rubric-level shifts. Across judge dimensions, terminology rises from 75.60 to 90.42 and target-language adherence from 82.88 to 96.15; terminology remains HexaBee-4B's lowest reference-assisted dimension. Against 4B IT, it wins, ties, or loses on 214, 54, and 32 reference-assisted examples respectively.
• Chinese-target scores stay lower in absolute terms, but the paper cautions that script, segmentation, reference phrasing, and the metric's character/word components affect cross-language comparisons, so this should not be read as a calibrated ranking of language difficulty.
• No component attribution. The paper states plainly that the comparison measures the complete recipe and does not isolate the effect of any individual training component.

Methodology in Plain English

The starting point is Gemma 3 4B IT. The authors adapt it in three steps.

First, they continue pretraining a LoRA adapter for one epoch on monolingual Wikipedia text in English, Persian, Arabic, Chinese, and Egyptian Arabic — 3,399,664 processed training examples drawn from raw source files containing 11,395,061 text rows. This teaches encyclopedic vocabulary and style but supplies no question-answer or translation pairs.

Second, they build a multilingual question-and-answer inventory from 2,025,730 Wikipedia-derived source rows, using Gemma 3 27B IT to expand Persian content into the other languages. From this inventory they create two kinds of training records under a shared objective: answering a question in a requested language, and translating an answer text between languages. System instructions specify the requested answer language, answer length or style, and translation direction, and the language a directive is written in is treated as separate from the language it requests. The same answer text can be the target of both task types. This phase continues the phase-1 adapter for one epoch on the combined mixture, and the adapter is then merged with Gemma 3 4B IT.

Third, they use a separately generated corpus. DeepSeek-V3.1 metadata identifies the teacher that produced source texts across topics, genres, registers, and complexity levels in all six languages, each paired with a synthetic translation into a different language. From 365,718 aligned pairs and a shuffled 90/10 split (329,146 raw training pairs, 36,572 test pairs), filtering yields 329,137 training pairs. Each retained pair becomes two training examples — forward and reverse — giving 658,274 records. A fresh LoRA adapter is trained for one epoch on the merged phase-2 checkpoint and merged to produce HexaBee-4B.

Evaluation uses 36,572 held-out synthetic examples across all 30 directions (1,173–1,276 examples per direction), scored by BLEU, chrF, chrF++, TER, and COMET (with the fixed wmt22-comet-da model), plus an archived DeepSeek judge in reference-assisted and reference-free modes on 300 matched items. Uncertainty is estimated with paired bootstrap resampling within each direction (10,000 replicates, seed 20260912). All three systems are evaluated on the same source texts and references, but with their own archived inference prompts, so the comparison includes prompt differences rather than isolating adaptation.

Why This Matters

For research, the paper offers a concrete, reproducible data-organization pattern — sharing a multilingual answer target between an answering task and a translation task — and a direction-level report of where it works and where it does not, including a documented metric disagreement. It also contributes a measured corpus composition, which the authors frame as the enabling artifact for future controlled ablation studies that they themselves did not run.

Real-world applications:

• Document and content localization for organizations that need the same material available in Arabic, Chinese, English, Hebrew, Persian, and Spanish.
• Cross-language information access, where a user asks a question in one language and wants a grounded answer in another.
• Assistive or internal tooling for multilingual customer support, where a response must both answer a question and be delivered in a specified language.
• Encyclopedic or knowledge-base translation, given that the training inventory is Wikipedia-derived and the first stage is encyclopedic continued pretraining.

For industry, the salient point is cost and size: a 4B-parameter model adapted with LoRA adapters in three short stages outperforms a public 12B model on this benchmark, which matters where inference budget and deployable model size are constraints. The caveat is equally relevant to practitioners — the evaluation is synthetic, the judge comes from the same teacher-service family, and inference prompts were not matched across systems.

Future Directions

• Controlled component attribution. The authors specify the experiment they did not run: keep the Wikipedia-adapted checkpoint, total training budget, final translation corpus, and inference prompts fixed while varying the intermediate answering/translation mixture.
• Evaluating the intermediate checkpoint to separate improvements that arise before and after translation specialization.
• Human-authored and out-of-domain evaluation. No human-authored translation benchmark is evaluated here; the paper names FLORES as a complementary setting for future validation, and notes that reference similarity and judge preferences may favor teacher-like outputs.
• Fixing the documented data defects and expanding coverage, including the 1,961 cross-language Spanish-target complete-answer records whose directives request English, the 6,742 empty input fields and 6,043 empty response fields, and the fact that complete-answer QA covers 27 cross-language directions while translation data cover all 30.

Target Audience

Researchers and engineers working on multilingual machine translation, parameter-efficient fine-tuning, and compact deployable models will get the most from this paper. It is also useful for practitioners choosing between a small adapted model and a larger off-the-shelf one, and for data-curation specialists interested in how task mixtures are constructed and audited. Readers should note the paper's own framing: it establishes a fully specified system and its translation evaluation on a synthetic test set, not causal claims about which training stage produces the gains, and not validated quality on human-authored or high-stakes translation.

Authors’ abstract

Multilingual translation requires a model to preserve meaning while producing text in a requested language. We present HexaBee-4B, a translator adapted from Gemma 3 4B IT through three stages: Wikipedia continued pretraining, joint multilingual answering and answer-text translation, and bidirectional translation specialization. The intermediate stage connects two tasks through shared multilingual answers: answering a question in the requested language and translating an answer into that language. It comprises 19.46 million answering records, including 5.76 million cross-language question-answer pairs, and 10.70 million answer-text translations. The final stage uses 658,274 forward and reverse synthetic translation examples. On 36,572 synthetic test examples spanning all 30 directions among Arabic, Chinese, English, Hebrew, Persian, and Spanish, HexaBee-4B achieves direction-macro chrF++ of 73.75, compared with 57.11 for Gemma 3 4B IT and 66.83 for Gemma 3 12B IT. BLEU, TER, COMET, and a matched model-judged subset support the same overall ranking. These results demonstrate improved translation across the six-language set, with chrF++ gains in every evaluated direction over both baselines.