Flexing in 73 Languages: A Single Small Model for Multilingual Inflection

AI summary

Overview

Research area: Natural Language Processing — computational morphology, specifically multilingual morphological inflection (generating inflected word forms from lemmas).

Technical level: Intermediate. Readers should know basic neural NLP concepts (Transformers, encoder–decoder models, train/dev/test splits), but the paper is largely empirical and its central claims are easy to follow without deep architectural background.

Scope: The paper presents a single compact Transformer model (5.69M parameters) trained jointly on inflection data from 73 languages, evaluates it against shared-task benchmarks and 73 Universal Dependencies treebanks, and introduces a new frequency-weighted, lemma-disjoint data-splitting procedure.

What This Paper Is About

Morphological inflection is the task of turning a base word (the lemma) plus a set of grammatical tags into the correct surface form — for example, English well + Degree=Sup becomes best. Existing inflection systems are typically trained one language at a time, which means deploying them for many languages requires managing, storing, and retraining dozens of separate models, and there is no open-source, lightweight system that can inflect previously unseen words across a wide language range (including Czech).

The authors' goal is to show that one small multilingual model can handle inflection for 73 languages at once, generalize to words never seen during training, and match or beat separately trained monolingual models.

Key Contributions

1. A single multilingual inflection model covering 73 languages. One Transformer with only 5.69M trainable parameters handles all parts of speech and all 73 languages, replacing the need for dozens of per-language models.

2. Strong empirical results on two independent evaluation setups. The model is tested on the SIGMORPHON 2022 and 2023 shared task benchmarks and on 73 Universal Dependencies treebanks lexically extracted by the authors themselves.

3. A new data-splitting procedure. The paper introduces a frequency-weighted, lemma-disjoint train–dev–test resampling method that combines two recent methodological recommendations: no lemma overlap between splits, and a realistic frequency distribution of items.

4. Open-source release. All code is publicly available, addressing what the authors identify as a gap in open-source, general-purpose multilingual morphological generation.

Main Findings

• The multilingual model beats monolingual models on most languages. Across the 73 Universal Dependencies corpora, the single multilingual model reaches a macro-average accuracy of 81.36%, versus 76.67% for separately trained monolingual models — an improvement of 4.69 percentage points.

• It is competitive on established shared-task benchmarks. On SIGMORPHON 2022 (feature-overlap setting) the multilingual model ranks 3rd on average across languages; on SIGMORPHON 2023 (lemma-disjoint setting) it ranks 1st, with an average accuracy of 85.6%.

• The model is extremely small. The multilingual model has 5.69M parameters; monolingual variants range from 3.12M to 3.69M. On the UD data, the naive "copy the lemma" baseline achieves only 45.27% average accuracy, showing the task is non-trivial and the model is doing real work.

• A small number of exceptions exist. The multilingual model is beaten by the copy baseline on 3 languages (Coptic, Maghrebi Arabic French/Arabizi, Naija) and by monolingual models on 7 languages (Czech, Estonian, Finnish, Latvian, Persian, Polish, Sanskrit). Notably, several of the losing languages (e.g., Persian, Finnish, Estonian, Latvian) are ones where the monolingual model benefits from concentrating capacity on a single, well-resourced morphology.

• Character-level input with a language-ID token works. Prepending a language identifier to the tag sequence lets the model disambiguate identical lemma strings across languages, and character-level tokenization keeps the vocabulary small and handles unseen words.

• Corpus size imbalance must be handled explicitly. Because the 73 datasets differ greatly in size, the authors used corpus upsampling with a temperature parameter (τ = 0.5) so that high-resource languages do not dominate training.

Methodology in Plain English

The researchers framed inflection as a sequence-to-sequence translation problem: the input is a lemma plus a string of morphological tags, and the output is the inflected form. Both input and output are processed character by character, which keeps the model small and lets it generalize to words it has never seen.

The architecture is a standard encoder–decoder Transformer trained from scratch — no pretrained language model, no massive corpus. It is deliberately kept small: 3 encoder/decoder layers for the monolingual models and 4 for the multilingual one, a layer dimension of 256, 4 attention heads, and modest dropout. The monolingual models are trained separately per language; the multilingual model is trained once on data pooled from all 73 languages.

To keep the multilingual model from being swamped by high-resource languages, the authors upsample smaller datasets and smooth the sampling distribution using a temperature value. A special language-ID token is added to each input so the model knows which language it is inflecting.

For data preparation, the authors extracted unique lemma–tag–form triples (with counts) from 73 Universal Dependencies treebanks. They also designed a new splitting method that keeps the test set free of lemmas seen during training (so the model is tested only on unseen words) while preserving the natural frequency distribution of items — an attempt to make evaluation match real-world conditions more closely than uniform random splits do.

Why This Matters

Impact on research. The paper questions the default assumption that inflection should be modeled one language at a time. It provides evidence that a single small model can share statistical strength across dozens of languages, and it contributes a concrete data-splitting recipe that future work can adopt or criticize. It also fills a documented gap: no open-source, lightweight inflection generator supporting out-of-vocabulary words across many languages, and none for Czech in particular.

Real-world applications.

• Grammar and spell checkers that need to suggest the correct inflected form of a word, including for words not in their dictionary.
• Language learning tools that generate correct forms for drills, conjugation tables, and declension exercises in many languages.
• Machine translation and text generation pipelines, where inflection generation is often a post-processing step that must produce grammatically correct surface forms.
• Low-resource language technology, where training one model per language is impractical but a shared multilingual model is deployable.

Industry relevance. The strongest industry argument is operational: managing 73 separate models means 73 deployment artifacts, 73 retraining jobs, and 73 sets of hyperparameters. A single 5.69M-parameter model that fits comfortably in memory and covers all languages drastically simplifies maintenance. For companies serving many language markets from a single backend, this is a meaningful cost and complexity reduction.

Future Directions

• Closing the remaining monolingual gaps. The multilingual model still loses to monolingual models on 7 languages (Czech, Estonian, Finnish, Latvian, Persian, Polish, Sanskrit). Understanding why — and whether more capacity, better balancing, or per-language adapters would fix it — is an open question.

• Handling noisy and idiosyncratic treebank data. The authors note that Universal Dependencies data is noisier than UniMorph, and the model's poor showing on Ancient Hebrew and Old French (where the copy baseline and monolingual model also fail) suggests data quality and orthographic quirks remain a problem.

• Extending to the inverse task and beyond generation. Morphological analysis (form → lemma + tags) is not covered here, and a complete morphological toolkit would need both directions, plus integration into existing tools such as UDPipe.

• Broadening language coverage and evaluating the split method further. The new frequency-weighted, lemma-disjoint split has not been tested by other researchers; reproducing and stress-testing it across more datasets and against other sampling schemes would clarify whether it should become standard practice.

Target Audience

This paper is most useful to NLP researchers and engineers working on computational morphology, multilingual modeling, and low-resource language technology, as well as practitioners who need to deploy inflection or generation systems across many languages. It will also interest researchers focused on evaluation methodology, since the data-splitting contribution is as much about how to measure inflection models as it is about the models themselves. Readers without any background in neural sequence-to-sequence models will find the results tables and the deployment argument accessible, but the methodology section assumes familiarity with Transformers.

Authors’ abstract

We present a compact, single-model approach to multilingual inflection, the task of generating inflected word forms from base lemmas to express grammatical categories. Our model, trained jointly on data from 73 languages, is lightweight, robust to unseen words, and outperforms monolingual baselines in most languages. This demonstrates the effectiveness of multilingual modeling for inflection and highlights its practical benefits: simplifying deployment by eliminating the need to manage and retrain dozens of separate monolingual models. In addition to the standard SIGMORPHON shared task benchmarks, we evaluate our monolingual and multilingual models on 73 Universal Dependencies (UD) treebanks, extracting lemma-tag-form triples and their frequency counts. To ensure realistic data splits, we introduce a novel frequency-weighted, lemma-disjoint train-dev-test resampling procedure. Our work addresses the lack of an open-source, general-purpose, multilingual morphological inflection system capable of handling unseen words across a wide range of languages, including Czech. All code is publicly released at: https://github.com/tomsouri/multilingual-inflection.