Systematicity between Forms and Meanings across Languages Supports Efficient Communication

AI summary

Overview

Research area: Computational linguistics and cognitive science — specifically efficient communication theory, linguistic complexity, and morphological typology (Natural Language Processing category, arXiv:2601.17181v1 [cs.CL], published 2026-01-23).

Technical level: Intermediate. The paper combines concepts from information theory (Information Bottleneck, mutual information, KL divergence), neural sequence-to-sequence learning, and linguistic typology. Readers with some background in computational linguistics will follow it; the core ideas are accessible without deep mathematics.

Scope: The paper proposes and tests a new learnability-based complexity measure (CETL) that captures systematic form-meaning structure in verb and pronoun paradigms across hundreds of languages, and compares it against the established Information Bottleneck model.

What This Paper Is About

Languages map meanings onto word forms in many different ways — English uses one word, you, for all second-person addressees, while Turkish distinguishes sen ("you, singular") from siz ("you, plural"). Existing efficient-communication accounts explain such variation as a trade-off between simplicity and accuracy, but they treat word forms as unanalyzed, atomic units and therefore cannot see regularity inside forms (for example, the Turkish prefix s- consistently marking second person). The paper's goal is to build a model whose measure of complexity reflects how learnable those internal form-meaning regularities are, and to test whether languages with more systematic structure are also more communicatively efficient.

Key Contributions

1. A new complexity measure based on learnability. The authors introduce Cross-Entropy Training Loss (CETL), which measures how quickly a neural sequence-to-sequence encoder learns to predict forms from meanings. Forms are read as character sequences rather than atomic units, so shared prefixes, suffixes, and other regularities lower the measured complexity. CETL is defined as the need-weighted cross-entropy loss summed over training epochs and divided by the maximum number of epochs.

2. A framework that unifies two research traditions. The model integrates efficient-communication research (Information Bottleneck) with language-evolution research on systematicity. It keeps the IB model's accuracy measure (based on Bayesian decoding) while replacing the IB complexity measure with CETL.

3. Extension beyond syncretism to partial formal overlap. The IB complexity measure can only detect syncretism (identical forms for different meanings). CETL additionally captures partial overlap, such as the systematic correspondence between grammatical second person and the prefix ta- in Arabic verbs.

4. A large-scale, typologically diverse evaluation. The study assembles verb paradigms in Semitic, non-Semitic Afro-Asiatic, Germanic, and Romance languages, and pronoun paradigms including the Pronoun Paradigms Database and family-specific pronoun sets (Semitic, Germanic, Afro-Asiatic, Romance, Slavic, Indo-Iranian, Altaic, and others), evaluated against systematically generated counterfactual permutations.

Main Findings

• Attested paradigms are more efficient than counterfactuals. Real verb and pronoun paradigms show lower CETL and higher accuracy than nearly all counterfactual structural and form-only permutations, with statistical significance across all domains. This supports Hypothesis 1 (Efficiency).

• More natural syncretism means lower complexity. For Afro-Asiatic verbs, CETL correlates positively with the unnaturalness score at ρ = 0.5745 (p < 2.2e-16). Averaged per-language correlations are 0.36 for the ppd pronoun domain and 0.82 and 0.88 for the pron and verb domains respectively. This supports Hypothesis 2 (Naturalness): paradigms with more natural syncretism patterns are more learnable.

• CETL beats IB on structural permutations. For ppd, CETL correctly identified 72.37% of structural permutations versus 4.82% for IB, with performance (correct minus incorrect) of 70.19 for CETL and 4.10 for IB. For pron: 90.12% vs 48.71% correct; performance 88.64 vs 44.18. For verb: 76.32% vs 41.03% correct; performance 72.55 vs 36.95.

• IB is blind to form-only permutations by design; CETL is not. On form-only permutations, CETL identified 65.81% (ppd), 89.21% (pron), and 78.23% (verb) of permutations as less efficient than attested paradigms, with performance scores of 31.63, 78.42, and 56.47. The IB model scored 0.00% correct and -100 performance in all three domains.

• CETL captures regularities IB cannot detect. Because form-only permutations preserve syncretism patterns exactly, IB cannot distinguish them from attested paradigms. CETL can, because these permutations may break consistent prefix or suffix patterns. The paper reports correlations exceeding 0.8 for PRON and VERB with naturalness, while IB shows no correlation in either domain.

• Four data domains with specific sizes. The evaluation covers ppd (561 languages, 93,845 total permutations), the detailed pronoun families (Pr_Sem: 45 languages / 16,437 permutations; Pr_Afro: 19 / 6,677; Pr_Ger: 30 / 10,649; Pr_Rom: 17 / 3,605; Pr_Slav: 20 / 6,063; Pr_IndoIran: 18 / 3,616; Pr_Altaic: 19 / 3,765; Pr_Other: 9 / 2,264), and the verb families (Verb_Sem: 56 languages / 19,700 permutations; Verb_Afro: 13 / 5,773; Verb_Ger: 32 / 8,875; Verb_Rom: 12 / 2,245).

Methodology in Plain English

The researchers treat language production as a communication problem. A speaker has something to say — a grammatical meaning such as "first person singular" — and picks a form to express it. The listener then tries to recover the meaning from that form.

Their model has three moving parts. First, a need distribution says how often each meaning must be communicated; the authors estimate this from corpus frequency, combining the need distribution from Zaslavsky et al. (2021b) with additional corpus frequency estimates for gender and dual/plural distinctions. Second, an encoder — an LSTM-based sequence-to-sequence network with two stacked LSTM layers in both encoder and decoder, trained with batch size 1 and dropout 0.5 for a maximum of 50 epochs — learns to map grammatical features onto forms, reading and writing forms character by character. Third, a decoder performs Bayesian inference to recover the intended meaning, which supplies the accuracy score (negative KL divergence between the intended meaning distribution and the reconstructed one).

The key insight is that complexity is measured by how fast the encoder learns. A paradigm with regular patterns (shared prefixes, consistent suffixes) is learned quickly and gets a low complexity score; a paradigm with arbitrary form-meaning pairings is learned slowly and scores high. This is CETL. Ten separate networks are trained for each original paradigm and five for each counterfactual paradigm, and results are averaged.

To test the model, the authors generate counterfactual paradigms by permuting the real ones. Structural permutations swap the contents of paradigm cells, changing which meanings share a form (altering syncretism). Form-only permutations swap surface forms while keeping the syncretism pattern unchanged. They also compute an unnaturalness score for each paradigm, counting how many feature values differ within each syncretic class (formula: sum over syncretism classes and feature categories of #c_f − 1). A paradigm where syncretic forms share many features is "natural"; one where they share none is "unnatural."

Finally, they check whether attested paradigms sit in a better region of the complexity-accuracy trade-off than their counterfactual alternatives, and whether performance beats the Information Bottleneck baseline. A model is counted as better than IB when it outperforms by at least 5% of the permutations for a given language.

Why This Matters

Impact on research. The paper connects two literatures that had developed separately: efficient-communication work on natural language lexicons, and language-evolution work showing that systematicity emerges in artificial languages as a response to simplicity and accuracy pressures. It argues that the IB model's treatment of forms as atomic units leaves a gap — it can see syncretism but not systematic internal structure — and shows that a learnability-based measure closes that gap. This gives a formal bridge between "systematicity" as studied in laboratory language-evolution experiments and "efficiency" as measured in typological data.

Real-world applications (potential, and beyond what the paper itself claims):

• Morphological analysis and generation systems for morphologically rich languages, where representing inflection as structured rather than memorized can reduce data requirements.
• Language documentation and description, offering a quantitative way to compare how systematic a paradigm is across under-documented languages.
• Second-language teaching and learning materials, since the measure is grounded in how quickly a learner-like network acquires a paradigm.
• Constructed language and orthography design, where designers may want forms that are learnable and systematically related rather than arbitrary.

Industry relevance. Any NLP system that processes morphologically rich languages — machine translation, speech recognition, information retrieval, text normalization — depends on how inflection is handled. A learnability-based complexity measure offers a principled criterion for deciding when a paradigm can be generated by rule-like patterns and when it must be memorized, which is directly relevant to data efficiency and to handling rare inflected forms.

Future Directions

• Extending beyond discrete paradigms. The authors state that CETL has only been tested on discrete, paradigmatically structured domains — verbal inflection and pronouns. Whether it applies to other semantic domains is uncertain, and they single out continuous domains like color as a fundamental challenge because they lack the discrete categorical structure the sequence-to-sequence architecture requires.

• Testing whether CETL is universally preferable or complementary to IB. The paper does not establish that CETL dominates IB in all respects. For instance, the authors do not replicate the fine-grained feature weight optimization of Zaslavsky et al. (2021b), leaving open whether CETL supports such analyses as effectively. They suggest the two measures may prove complementary, suited to different applications.

• Building an explicit Pareto frontier. Unlike some prior work in the efficient communication literature, the paper does not provide a Pareto frontier of maximally efficient paradigms. Doing so would require specifying all possible counterfactual paradigms, including all forms compatible with each language's phonology and phonotactics. The permutation-based approach generates structured counterfactuals but does not exhaustively sample the space, limiting strong optimality claims.

• Broadening language coverage and paradigm representation. The study deliberately includes only one or two paradigms per language and excludes irregular verbs and stem-lexeme interactions in order to control for other complexity models; the authors evaluate robustness to this choice but the design leaves room to test how far the conclusions extend under more variable morphological data.

Target Audience

This paper is most useful to computational linguists and cognitive scientists working on efficient communication, linguistic complexity, or morphology; to typologists interested in quantitative comparisons of paradigm structure; and to NLP researchers working on morphologically rich or low-resource languages who want a principled complexity criterion. Researchers in the language-evolution and artificial-language-learning community will find the connection to systematicity especially relevant. A reader with basic familiarity with information theory and neural sequence-to-sequence models will get the most from it, though the central argument can be followed without the mathematics.

Authors’ abstract

Languages vary widely in how meanings map to word forms. These mappings have been found to support efficient communication; however, this theory does not account for systematic relations within word forms. We examine how a restricted set of grammatical meanings (e.g. person, number) are expressed on verbs and pronouns across typologically diverse languages. Consistent with prior work, we find that verb and pronoun forms are shaped by competing communicative pressures for simplicity (minimizing the inventory of grammatical distinctions) and accuracy (enabling recovery of intended meanings). Crucially, our proposed model uses a novel measure of complexity (inverse of simplicity) based on the learnability of meaning-to-form mappings. This innovation captures fine-grained regularities in linguistic form, allowing better discrimination between attested and unattested systems, and establishes a new connection from efficient communication theory to systematicity in natural language.