SumTablets: A Transliteration Dataset of Sumerian Tablets

AI summary

Overview

Research area: Natural Language Processing applied to a low-resource, ancient language — specifically machine transliteration of Sumerian cuneiform into Latin-script transliteration — sitting at the intersection of NLP, dataset construction, and digital Assyriology.

Technical level: Intermediate. The dataset construction and preprocessing portions are accessible to non-specialists; the neural baseline assumes familiarity with encoder–decoder transformers, masked language modeling, and character-level evaluation metrics.

Scope: The paper releases a 91,606-tablet parallel glyph–transliteration dataset and uses it to define the Sumerian transliteration task and establish two baselines.

What This Paper Is About

Assyriologists record their interpretation of a cuneiform tablet as a transliteration — a Latin-script rendering of each glyph — and decades of digital Assyriology projects (ETCSL, CDLI, Oracc) have made large numbers of these transliterations publicly available. What was missing was a dataset that pairs each transliteration back with a digital (Unicode) representation of the glyphs actually on the tablet, which is what a model needs in order to learn the mapping. The paper builds that paired dataset from Oracc data and then tests whether modern multilingual language models can perform the glyph-to-transliteration conversion.

Key Contributions

1. A new dataset, SumTablets: 91,606 Sumerian cuneiform tablets paired with their transliterations, totaling 6,970,407 glyphs, with IDs, period, and genre metadata, released as a Hugging Face Dataset under CC BY 4.0 with open-source preparation code on GitHub.
2. A standardization pipeline: preprocesses and normalizes Oracc transliterations, removes editorial annotations, encodes structural information (surfaces, line breaks, breakage, rulings, columns, blank space) as special tokens, and maps readings back to Unicode glyphs using ePSD2 and the Oracc Sign List.
3. A task definition and evaluation protocol: frames transliteration as sequence-to-sequence conversion rather than token classification, with a held-out test set stratified by time period and character-level chrF as the metric.
4. Two baselines: a weighted dictionary-sampling approach (chrF 61.22) and a fine-tuned XLM-R encoder–decoder model (chrF 97.54; the abstract states 97.55), described as the first automatic Sumerian transliteration model.

Main Findings

• Preprocessing retains nearly all data: of the readings in the source transliterations, 6,724,498 (99.93%) were successfully mapped to glyph names, and 6,638,081 (99.96%) of glyph names were mapped to Unicode.
• The neural model substantially outperforms dictionary lookup: the fine-tuned XLM-R model reaches an average chrF of 97.54 (the abstract reports 97.55), against 61.22 for the dictionary baseline.
• Performance varies strongly by period: dictionary scores range from 37.70 (Old Babylonian) to 73.72 (Early Dynastic I-II); neural scores range from 89.79 (Neo-Assyrian) to 98.46 (Ur III).
• Performance varies strongly by genre: the neural model scores highest on Administrative texts (98.14) and lowest on Liturgy (77.68); the dictionary baseline scores lowest on Literary (37.73) and highest on Unknown genre (69.84).
• Administrative texts dominate the corpus: the period distribution is led by Ur III (71,116 train / 3,951 val / 3,951 test tablets), and the genre distribution by Administrative (77,193 train / 4,259 val / 4,291 test tablets).
• Glyph polyvalency is a central challenge: the average number of distinct readings per glyph, weighted by glyph frequency, is 22.17, which is why a context-blind dictionary baseline performs poorly.
• Genre affects difficulty for intuitive reasons: administrative texts are formulaic and abundant in training data, while liturgical, letter, and literary texts differ in style, form, and vocabulary — and are also among the hardest for human experts.
• Inconsistent transliteration conventions hurt accuracy: disagreements such as "saŋ" versus "sag" for the same nasal sound fragment the patterns a model can learn.
• Names are a source of error: error analysis found the model often predicts a valid reading for a glyph in a name but a different one than the reference transliteration records.

Methodology in Plain English

The authors start from JSON transliterations published through ePSD2, a project that aggregates Oracc data. These transliterations were typed by scholars over decades under evolving conventions, so the first job is cleaning: parse the recursive document structure into a single string, strip out editorial annotations (square brackets mark reconstructed broken text, angle brackets mark supplied or excised graphemes, and so on), and store what is left in a standardized form. Structural features that carry meaning — the start of a surface, line breaks, breakage, scribal rulings, column starts, blank space — get their own special tokens so that the glyph sequence and the transliteration sequence line up in parallel.

Next, each transliteration is split into words and then into individual glyph readings. Each reading is looked up in dictionaries built from ePSD2 and the Oracc Sign List to recover the glyph's conventional name (glyph names are uppercase versions of one of their readings), and each glyph name is then converted to its Unicode cuneiform character. Readings or glyphs that cannot be resolved become <UNK>.

For evaluation, the corpus is split 90%/5%/5% into train, validation, and test, stratified by period so that each partition reflects the same range of time periods; lexical texts are removed before splitting and added back to the train set afterward. Two baselines are compared. The dictionary baseline samples a reading for each glyph in proportion to how often that reading occurs. The neural baseline retrains XLM-R's SentencePiece tokenizer twice — once for glyphs (632-token vocabulary) and once for transliterations (1,024-token vocabulary), each including eleven special tokens — then initializes an encoder and a decoder separately from a 279-million-parameter XLM-R checkpoint. Training proceeds in three stages: masked language modeling on glyphs alone (50 epochs), then encoder–decoder training with the encoder frozen (2 epochs), then the full model with the encoder unfrozen (4 epochs). Because tablets can exceed the 128-token maximum sequence length, longer tablets are chunked at newline boundaries using progressively smaller groupings of lines. Non-administrative examples are up-sampled by a factor of 5 for the first two epochs and 3 for the rest. Generation uses beam search with a beam size of 5.

Why This Matters

Impact on research. SumTablets makes the Sumerian transliteration task tractable for NLP research for the first time by supplying parallel inputs and targets in a format a standard library can load. It also gives a defined metric, a split, and reference baseline numbers so future results are comparable. Because Sumerian is both a low-resource language and a language isolate, it is a useful stress test for whether cross-lingual pretrained models transfer meaningful linguistic understanding to a language they were never trained on.

Real-world applications.

• Web-based tools that let Assyriologists generate a draft transliteration and verify it, rather than transliterating every tablet manually from scratch.
• Targeted review of existing published transliterations, flagging passages where a model's reading differs from the recorded one.
• A building block toward a fuller Sumerian translation pipeline, since transliteration is a prerequisite step.
• Classification or retrieval workflows over tablet metadata, periods, genres, and texts, benefiting from a standardized, easily loadable corpus.

Industry relevance. The paper is primarily a humanities-research contribution with open licensing (CC BY 4.0) and open code, so direct commercial impact is limited. Its methods — retraining tokenizers for a script with no existing tokenization support, staged encoder–decoder fine-tuning from a multilingual checkpoint, and oversampling to correct a heavy domain imbalance — are directly transferable to other domain-specific and low-resource transliteration or transcription problems.

Future Directions

• Zero- and few-shot evaluation. The paper trains fully supervised with access to the entire training set and explicitly does not study zero- or few-shot cross-lingual transfer, which the authors suggest as a future benchmark use of SumTablets.
• Training from scratch. The authors note they did not study a model trained from scratch on SumTablets, leaving this as an open comparison point.
• Better dictionary baselines. The authors acknowledge their dictionary baseline is very simple and that an N-gram model would be a stronger point of comparison.
• Handling unrepresented glyphs and orthographic variation. Complex compound glyphs outside Unicode are currently collapsed to <UNK>; the authors suggest unique placeholder identifiers, and note that flattening time-varying orthography into a single Unicode representation may discard meaningful distinctions.
• Expert evaluation of name readings. Future work is proposed to determine whether the model's alternative readings for glyphs in names are more or less plausible than the versions recorded in the reference transliteration.

Target Audience

Researchers in NLP working on low-resource, historical, or non-Latin-script languages; computational linguists interested in cross-lingual transfer and transliteration; and digital Assyriologists or Sumerologists who want to build or use automated transliteration tooling. The dataset description and baseline description are readable by graduate students entering either field, while the training details and analysis are aimed at readers comfortable with transformer encoder–decoder models. Practitioners looking for a worked example of adapting a multilingual checkpoint to a completely unsupported script will also find the pipeline useful.

Authors’ abstract

Sumerian transliteration is a conventional system for representing a scholar's interpretation of a tablet in the Latin script. Thanks to visionary digital Assyriology projects such as ETCSL, CDLI, and Oracc, a large number of Sumerian transliterations have been published online, and these data are well-structured for a variety of search and analysis tasks. However, the absence of a comprehensive, accessible dataset pairing transliterations with a digital representation of the tablet's cuneiform glyphs has prevented the application of modern Natural Language Processing (NLP) methods to the task of Sumerian transliteration. To address this gap, we present SumTablets, a dataset pairing Unicode representations of 91,606 Sumerian cuneiform tablets (totaling 6,970,407 glyphs) with the associated transliterations published by Oracc. We construct SumTablets by first preprocessing and standardizing the Oracc transliterations before mapping each reading back to the Unicode representation of the source glyph. Further, we retain parallel structural information (e.g., surfaces, newlines, broken segments) through the use of special tokens. We release SumTablets as a Hugging Face Dataset (CC BY 4.0) and open source data preparation code via GitHub. Additionally, we leverage SumTablets to implement and evaluate two transliteration baselines: (1) weighted sampling from a glyph's possible readings, and (2) fine-tuning an autoregressive language model. Our fine-tuned language model achieves an average transliteration character-level F-score (chrF) of 97.55, demonstrating the immediate potential of transformer-based transliteration models in allowing experts to rapidly verify generated transliterations rather than manually transliterating tablets one-by-one.