BIRD: Bronze Inscription Restoration and Dating

AI summary

Overview

Research area: Natural Language Processing applied to ancient Chinese epigraphy, specifically the restoration and chronological dating of Chinese Bronze Age inscriptions.

Technical level: Intermediate. Readers need familiarity with masked language modeling, Transformer encoders (BERT/RoBERTa), and domain/task-adaptive pretraining, but the paper explains its paleographic motivations clearly.

Scope: The paper introduces BIRD, a 41k-token fully encoded bronze inscription corpus with chronological labels, plus Glyph Net (1,078 grapheme–allograph pairs), and shows that allograph-aware masked language modeling improves restoration while glyph-biased sampling improves dating.

What This Paper Is About

Bronze inscriptions from early China (21st–3rd c. BCE) are fragmentary and hard to date, and prior AI work on them has been almost entirely computer vision rather than NLP. The authors build the first fully encoded, deduplicated, filtered corpus of these inscriptions and test whether adapting pretrained language models to this low-resource, allograph-heavy domain can automatically restore damaged characters and predict when an inscription was made. The core tension they address is that one task (restoration) benefits from treating allographic variants as the same grapheme, while the other (dating) benefits from preserving glyph distinctions as chronological evidence.

Key Contributions

1. BIRD dataset: The first fully encoded bronze inscription dataset for NLP, comprising 41k tokens with dynasty- and period-level chronological labels. It is the only resource in the comparison that is available, deduplicated, filtered, fully encoded, and chronologically annotated (unlike Jihewang, AS DABII, and AS Lexicon, which are partial on encoding and lack the other properties).
2. Glyph Net (GN): A resource of 1,078 grapheme–allograph pairs compiled from Shang, Western Zhou, and Eastern Zhou studies, which clusters allographic forms into glyph families using the principle of mutual substitutability.
3. Allograph-aware modeling framework: A masked language modeling pipeline combining domain-adaptive pretraining (DAPT) on a Pre-Qin corpus, task-adaptive pretraining (TAPT) on BIRD, GN-based regularization, and a glyph-biased sampling strategy.
4. Dual-task evaluation: Systematic experiments on restoration (Exact@K and Family@K) and dating (dynasty-level, period-level, and hierarchical metrics) across BiLSTM, SVM, mBERT, XLM-RoBERTa base/large, and SikuRoBERTa backbones.

Main Findings

• Restoration is led by SikuRoBERTa: It achieves the best performance on five of six restoration metrics, including 49.47 Exact@1 and 73.07 Family@10, outperforming BiLSTM by +10.5 percentage points (Exact@1) and +10.6 percentage points (Family@10).
• BiLSTM wins one metric: BiLSTM leads on Family@1 with 57.41.
• In-domain pretraining matters: Multilingual PLMs lag SikuRoBERTa by 4–6 percentage points on Exact@1, which the authors attribute to the advantage of in-domain pretraining.
• Dating is led by SikuRoBERTa: Best overall dynasty accuracy of 86.42 and macro-F1 of 77.83, with the highest hierarchical dynasty accuracy at 84.21; mBERT trails by 1–4 points, while larger multilingual encoders are less competitive.
• Period dating is harder than dynasty dating: This holds across the reported results. XLM-Large surpasses SikuRoBERTa on Hier-Per F1 but lags elsewhere.
• Different modules help different tasks: For restoration, GN is the most consistent contributor — on SikuRoBERTa it achieves the highest Exact@K and Family@K, averaging 58.3 across metrics, versus Bias (57.8) and no adaptation (56.5). For dating, the trend reverses: glyph-biased sampling averages 68.9 across metrics, which is 2.4 and 1.3 percentage points higher than GN (66.5) and GN+Bias (67.6) respectively.
• Takeaway: Restoration benefits from modeling allographic equivalence, whereas dating exploits diachronic differentiation.
• Error patterns: Restoration is strongest in formulaic segments and on nouns denoting vessels, temporal adverbs, and modal particles; errors default to frequent templates and confusion among semantically related nouns, verbs, or numerals. Dating misclassifications concentrate in the Spring and Autumn and Warring States periods, and class imbalance skews errors toward the Western Zhou, though severe cross-era errors remain rare.
• Out-of-sample case study: On the mid–Western Zhou Hu Ding inscription (CCYZBI.02838A/B), with all Hu Ding entries excluded from BIRD before training, SikuRoBERTa (GN) achieved Exact@1 of 50.00% (11/22), Exact@5 of 59.09% (13/22), and Exact@10 of 68.18% (15/22) under parallel prediction; greedy decoding gave comparable coverage with lower accuracy.

Methodology in Plain English

The authors start from the observation that expert restoration of a damaged inscription looks a lot like a fill-in-the-blank exercise: the paleographer reads the surrounding text and proposes what is missing. That maps naturally onto masked language modeling, where a model predicts a hidden token from its context.

To make this work, they first build a usable corpus. They take scholarly transcriptions, convert everything into machine-readable text with three token categories — identifiable characters, unreadable marks (□), and undeciphered forms ([UNK]) — then remove inscriptions of one character or fewer (6,078 of 17,547 entries in AS DABII), drop duplicate inscriptions that recur across vessels, and revise clerical transcriptions and dates using recent philological work.

Because the corpus is small, they borrow strength from a larger body of contemporaneous Pre-Qin writing: 40 works in 11 categories totaling 2.09M tokens. The model first reads this general Pre-Qin material (domain-adaptive pretraining, with the bottom six layers frozen for ten epochs) and then adapts to the inscriptions themselves (task-adaptive pretraining, all layers unfrozen).

The allograph problem is handled by Glyph Net. Since the same grapheme can appear in many different graphical forms, the authors link forms into families. During restoration, the model is trained to spread probability across the members of the correct family rather than committing to a single form, and evaluation credits a prediction if any family member appears in the top-K. During dating, the opposite logic applies: a glyph-biased sampling scheme makes the model pay more attention to glyph tokens, because variant forms themselves carry chronological signal.

Because inscriptions are so short that ordinary random masking can strip away nearly all context, they use a stride-based masking scheme that masks every s-th non-boundary character, so sequences of length at most s lose at most one token. Stride values were tuned per backbone via Bayesian hyperparameter search with Weights & Biases. Hyperparameters reported include a learning rate of 0.00005 for mBERT, XLM-Base, and XLM-Large and 0.00012 for SikuRoBERTa; 60 epochs for mBERT and 40 for the others; batch size 32; attention and hidden dropout 0.1; strides of 10 or 12; mlm_prob 0.2; and weight decay 0.01.

Why This Matters

Impact on research: This is the first NLP-ready dataset and baseline framework for Chinese bronze inscriptions, moving the field beyond image-based computer vision toward language modeling. It also demonstrates a general pattern for low-resource epigraphy: normalize where equivalence helps, preserve variation where distinction carries meaning. The BIRD resource and Glyph Net make restoration and dating directly comparable across model families.

Real-world applications:

• Assisting paleographers and museum researchers in proposing candidate readings for damaged or undeciphered inscription positions, as demonstrated on the Hu Ding vessel.
• Rapid preliminary chronological classification of newly excavated or unprovenanced bronzes, providing an independent textual signal alongside typology and archaeology.
• Cataloging and indexing digitized collections, where consistent glyph-family treatment could improve search and retrieval across variant character forms.
• Educational and cultural-heritage tools that make early Chinese textual sources more accessible to non-specialists.

Industry relevance: The work is a case study in domain adaptation for extremely low-resource, specialized text. The same recipe — DAPT on a broader contemporaneous corpus, TAPT on the target domain, and knowledge-graph-style normalization of surface variation — transfers to other specialist domains such as historical legal records, dialectal text, clinical notes, and any setting where a small corpus is riddled with near-duplicate surface forms.

Future Directions

• Address data sparsity and long-tail imbalance: BIRD still suffers from both, which constrains generalization to rare forms.
• Strengthen glyph-level modeling: Different characters may not consistently represent the same word, and Glyph Net currently relies only on inductive bias at the family level. Stricter palaeographic constraints, a larger knowledge base of loan characters, and explicit modeling of diachronic allograph distributions are all proposed, since the system can produce semantically plausible but orthographically inappropriate predictions.
• Add phonological supervision: Bronze and other early Chinese inscriptions frequently use loans, but sound-based substitution is invisible to a token-only model. Phonetic series embeddings are suggested as a remedy, following phoneme-aware strategies used in non-Latin scripts.
• Exploit partial and multimodal evidence: Token-level MLM cannot use subcomponents of a partially damaged graph; structure-aware encodings such as Ideographic Description Sequences could enable component-conditioned modeling. Separately, vessel shape, decorative motifs, and casting techniques provide independent chronological evidence that the current framework ignores.
• Explore generative architectures: The experiments relied on BERT- and RoBERTa-based backbones for efficiency; more recent generative models might better capture long-range dependencies and support free-form restoration.

Target Audience

Digital humanities and computational linguistics researchers working on ancient or historical text; NLP practitioners interested in low-resource domain adaptation and lexical normalization; paleographers and sinologists curious about what current models can and cannot recover from bronze inscriptions; and machine learning engineers looking for a worked example of combining knowledge-graph structure with masked language modeling. Readers without Chinese epigraphy background can follow the method, but the case study and error analysis will be most meaningful to those familiar with the material.

Authors’ abstract

Bronze inscriptions from early China are fragmentary and difficult to date. We introduce BIRD(Bronze Inscription Restoration and Dating), a fully encoded dataset grounded in standard scholarly transcriptions and chronological labels. We further propose an allograph-aware masked language modeling framework that integrates domain- and task-adaptive pretraining with a Glyph Net (GN), which links graphemes and allographs. Experiments show that GN improves restoration, while glyph-biased sampling yields gains in dating.