Cross-lingual Biography Enrichment via Claim Extraction and Alignment

AI summary

Overview

Research area: Natural Language Processing, specifically cross-lingual text generation, Wikipedia/NLP resource construction, and claim-level factuality.

Technical level: Intermediate. The paper assumes familiarity with LLM prompting pipelines, claim extraction/decomposition frameworks, and evaluation metrics, but the task framing itself is straightforward.

One-sentence scope: The paper defines and benchmarks the task of enriching an existing English Wikipedia biography using facts taken from a non-English Wikipedia biography of the same person, and proposes a claim-based pipeline that extracts, aligns, and selects non-English-side facts before rewriting.

What This Paper Is About

Wikipedia coverage is uneven across languages: the same person may be documented in far more detail in a non-English edition than in English, particularly for women from non-English-speaking contexts. The paper asks whether an existing English biography can be enriched with information that appears only in its French, Chinese, or Azerbaijani counterpart, without introducing unsupported or contradictory statements. The authors frame this as an explicit evidence-selection problem: rather than asking a model to read a long foreign-language article, they extract claims from both articles, align them, keep only the claims that the English article does not already cover, and rewrite the English article using those claims.

Key Contributions

1. A new task formulation. The paper introduces cross-lingual biography enrichment, defined as generating an English biography that preserves the content of the original English biography while incorporating additional information supported by a paired non-English biography, focusing on women from non-English-speaking contexts.

2. Three new benchmarks. The authors release CLAW-4L (300 cross-lingual biography pairs from English into French, Chinese, or Azerbaijani, with claim annotations), CLAW-4L-CX (600 sentences with manually annotated English claims), and CLAW-4L-RC (600 English claim pairs annotated with fine-grained semantic relations).

3. A claim-based enrichment framework. The pipeline maps both biographies into a shared English claim space, uses retrieval plus an LLM classifier to align claim pairs, discards claims already covered by English, and rewrites the English biography using the selected enrichment claims as structured evidence.

4. A comparative evaluation of three evidence representations. The paper evaluates raw non-English input, machine-translated non-English input, and selected enrichment claims across three generators and three languages, with automatic claim-level metrics plus a small human writing-quality check.

Main Findings

• Non-English biographies are substantially richer. Across CLAW-4L, non-English biographies average 3.4 times more tokens, 3.4 times more sentences, and 2.9 times more claims than their English counterparts. The longest non-English biography reaches 28,420 tokens and the largest extracted claim set reaches 1,363 claims, which the authors highlight as a context-length challenge.

• Most non-English claims are genuinely additive. Of non-English-side claims, only 13.9% are already covered by English (exact matches or English-more-specific alignments), 26.2% add detail to partially aligned English claims, 11.4% are classified as conflicts, and 48.6% have no relevant English-side counterpart. In total, 86.2% of non-English-side claims are selected as enrichment evidence.

• Adapted Claimify is the strongest claim extractor. Among five LLM-based frameworks adapted to the cross-lingual setting and instantiated with GPT-5.1, X-Claimify performs best across all four languages, with average Aligned-F1 of 73.72 and Exact-Aligned-F1 of 71.18. X-DnDScore is weakest, at 44.40 and 16.65.

• A 9B classifier nearly matches a closed model at claim alignment. In the text-only setting, Qwen3.5-9B achieves 95.5 ARC, 89.5 Align-FG, and 93.2 Overall, close to GPT-5.1 at 95.9, 90.8, and 93.3. Adding structured claim fields (subject, predicate, object) does not consistently help and can introduce parsing noise, so the text-only Qwen3.5-9B classifier is used downstream.

• Claim-based evidence gives the best supported-additions-versus-hallucination trade-off. Averaged over all languages and generators, translation raises average #Valid from 15.89 to 19.05 while hallucination stays nearly flat (29.59 versus 29.49 for raw). Claim-based evidence achieves the highest average #Valid (20.47) with substantially lower hallucination (23.54).

• Translation beats claims in exactly one setting. Azerbaijani with Mistral is the only language–generator combination where translation outperforms claim evidence on the Bal. score. The authors attribute this to error propagation in upstream claim extraction for the lower-resource language.

• Language and generator effects are substantial. Performance is stronger for French and Chinese than for Azerbaijani. Mistral-3.2-24B-it achieves the highest Bal. score in every language–evidence setting, largely driven by lower hallucination rates while remaining competitive on supported additions.

• No evident writing-quality penalty from using claims. In a human sanity check on 45 Mistral-generated biographies rated by three computer-science Ph.D. researchers on a 1–5 Likert scale, claims score highest on Readability/Fluency (3.933) and Wikipedia-Style Writing (3.822); raw input is slightly higher on Coherence/Integration (3.911 versus 3.867 for claims); translation scores 3.756, 3.733, and 3.689.

• Qualitative errors are concentrated in specific fact types. In a manual inspection of 18 Mistral-generated biographies from six controlled cases, missing claims mainly involved long-tail works, institutional roles, and event metadata, while remaining hallucinations included incorrect temporal, geographic, and educational details, unsupported over-specific enumerations, and repetitive generation in one translation-based case.

Methodology in Plain English

The researchers start by collecting pairs of Wikipedia biographies about the same woman from Wikidata, requiring a country of citizenship and sitelinks in both English and the target non-English language. To favor cases where the non-English article is richer, they rank candidates by a language-calibrated token ratio (normalizing raw token counts with language-specific inflation factors to reduce tokenizer bias), then pick 100 biographies per language with coarse occupation-aware balancing over artists, scientists, athletes, and politicians.

To compare claims across languages, both biographies are first broken into English-language claims. Five existing claim extraction frameworks are adapted to accept either English or non-English input and output English claims; Claimify works best. To check whether two claims say the same thing or differ, the team trains an LLM classifier on a two-level label scheme: aligned, contradicted, or not relevant at the top level, with aligned pairs further split into exact equivalence, one-sided enrichment, and mutual enrichment.

Selecting enrichment evidence is a two-step filter. For efficiency, each non-English-side claim retrieves the top-5 English claims by All-MPNet-Base-v2 cosine similarity above a 0.7 threshold; pairs that fail retrieval are treated as not relevant. The classifier then judges the retained pairs. Claims already covered by English (exact matches, or English-more-specific) are discarded; non-English-additive alignments, conflicts, and unmatched claims are kept as enrichment candidates.

Generation is then compared in three formats — raw non-English input, machine-translated input, and selected claims — using the same three open-weight instruction-tuned generators (Qwen3.6-27B, Gemma-4-31B-it, Mistral-3.2-24B-it). Long articles are handled incrementally at section level, with the English input capped at 4,096 tokens and the non-English context at 2,048 tokens per step. Evaluation extracts claims from each generated biography with X-Claimify (using Gemma-4-31B-it as backbone) and classifies each generated claim against a reference pool formed by the union of English and non-English claims, using contradiction priority. The main reported score, Bal., ranges from 0 to 100 and equally weights normalized supported claim growth and factual reliability.

Why This Matters

This work moves Wikipedia enrichment away from generating biographies from infoboxes or web retrieval and toward using human-written, curated encyclopedic articles in other languages as evidence. It also makes evidence selection an explicit, inspectable step before generation, rather than hoping a model will implicitly align two long texts across languages — a design that supports intermediate evaluation and reduces context burden.

Real-world applications:

• Wikipedia editing assistance. Surfacing specific, source-backed claim recommendations for editors working on underspecified English articles about women from non-English-speaking regions.
• Editorial triage of cross-language conflicts. Flagging cases where an English and a non-English edition disagree, so a human editor can reconcile rather than the system silently overwriting.
• Multilingual knowledge-base curation. Producing interpretable claim-level alignment trails that can be audited, rather than opaque paragraph-level merges.
• Benchmark reuse for multilingual factuality tooling. The three released benchmarks (biography pairs, cross-lingual claim extraction, claim-pair relation classification) can be reused to evaluate extraction and alignment components independently of any generation system.

Industry relevance. Any organization maintaining multilingual knowledge resources — search, virtual assistants, retrieval-augmented generation, fact verification — faces the same asymmetry this paper addresses. The finding that explicit claim selection cuts hallucination while increasing supported additions (23.54 versus 29.59 for translation) is directly relevant to pipelines that must add information from foreign-language sources without degrading reliability, and the near-parity of a 9B open classifier with GPT-5.1 on claim alignment matters for cost-sensitive deployments.

Future Directions

• Generate–verify–revise loops. The authors propose verifying each generated claim against the input evidence (using methods such as FactScore, VeriScore, FactCheck-GPT, or MiniCheck) and removing or correcting unsupported claims before a final rewriting step, optionally retrieving independent corroboration from knowledge bases, citations, and additional language editions.

• Stronger claim extraction and alignment for lower-resource languages. The Azerbaijani results show that upstream extraction misses useful evidence, which then limits the claim-based generator. Improving these components is presented as the prerequisite for reliable enrichment in low-resource settings.

• Bidirectional and many-to-many enrichment. The current formulation treats English as the generation target. The authors point to aligning claims from multiple language editions to support expansion in lower-resource Wikipedia editions instead.

• Broader benchmark scope and better human evaluation. Extending beyond women biographies in French, Chinese, and Azerbaijani, and replacing the current descriptive sanity check (15 biography pairs, one generator, three annotators, with method labels visible) with a more rigorous, significance-tested assessment of editorial quality.

Target Audience

Researchers and practitioners in multilingual NLP, low-resource language technology, and knowledge-base construction; Wikipedia and Wikimedia tooling developers; and teams building retrieval-augmented or cross-lingual generation systems who need factual grounding and interpretable evidence selection. The CLAW-4L benchmarks are also directly useful to anyone evaluating cross-lingual claim extraction or fine-grained claim relation classification, and the paper's task framing is accessible to readers with intermediate NLP background.

Authors’ abstract

English Wikipedia is often treated as the default encyclopedic source, yet non-English Wikipedia editions can contain richer locally grounded information for long-tail figures. We study cross-lingual biography enrichment: enriching an existing English biography with facts supported by a non-English biography about the same person. Focusing on women from non-English-speaking contexts, we introduce \textsc{CLAW-4L}, a benchmark consisting of 300 Wikipedia biography pairs linking an English biography with its French, Chinese or Azerbaijani counterpart, along with claim annotations and a fine-grained claim-pair relation corpus. We propose a claim-based enrichment framework that extracts English claims from both biographies, aligns them to identify enrichment evidence from the non-English biography, and rewrites the English biography using the selected claims. Our results show that non-English Wikipedia biographies provide valuable evidence for improving English biography coverage, while lower-resource settings remain challenging.