Thought without systematicity? Evaluating reasoning models on rule induction tasks

AI summary

Overview

• Research area: Natural Language Processing / evaluation of reasoning models, drawing on cognitive science paradigms (rule induction, systematicity, the language-of-thought hypothesis).
• Technical level: Intermediate. The paper combines a formal definition of systematicity with an empirical benchmark study, but the core ideas are explained through concrete examples (colored shapes, pseudowords like "dax" and "wudsy").
• Scope in one sentence: The paper formalizes systematicity as invariance of task-solving ability under structure-preserving task transformations, and empirically tests whether current reasoning models maintain that invariance across four families of synthetic rule induction tasks.

What This Paper Is About

Systematicity is the cognitive principle that understanding one concept should entail understanding its close variations — if you can learn that all red squares are "wudsy," you should equally be able to learn that all blue circles are "wudsy." The authors ask whether reasoning models exhibit this property: if a model can solve a rule induction task, does it also solve structurally equivalent versions of that same task that differ only in surface details? To answer this, they build families of synthetic tasks with compositional structure and generate isomorphic variants of each task, then check whether model performance holds up.

Key Contributions

1. A formalization of systematicity measurable over rule induction tasks. The authors define a task as a tuple T = (f, S, Q) — a hidden rule f: X → Y, a support set S of input-output examples, and a query set Q of inputs to be predicted — and define systematicity as invariance of task-solving ability under isomorphisms: invertible, structure-preserving mappings that can only alter a task's surface form.

2. A taxonomy of systematic task variations. Four transformation types are identified and used: symbol substitution, feature rebinding, constituent permutation, and example reordering.

3. Three metrics for quantifying systematicity. For K = 5 task variants per task-invariance type, the paper reports "solve any of K variations" (max), "fraction of variations solved" (mean), and "solve all K variations" (min), visualized as bullet charts.

4. Four adapted rule induction task families for systematicity evaluation: grammar-based instruction learning (Lake and Baroni, 2023), a symbolic version of Raven's progressive matrices (Schug et al., 2025), program induction over integer sequences (Rule et al., 2024), and Boolean category learning (Piantadosi et al., 2016).

5. An empirical evaluation across these task families, including an investigation of the impact of reasoning effort and sampling noise.

Main Findings

• Ability does not imply systematicity. Despite being able to correctly solve a task, models often fail on structurally equivalent variants of the same task.

• A systematicity gap on three of four task families. There is a notable gap between the ability to solve any task variation and the ability to solve all task variations on the grammar-based instruction-learning, Raven's-progressive-matrices, and program-induction families.

• Boolean category learning is an exception. Almost all models are perfectly systematic on the Boolean category learning task family.

• Task-solving ability can be context-bound. The authors conclude that many model behaviors lack systematicity, making it difficult to robustly establish the cognitive abilities of reasoning models beyond the particular contexts they were evaluated in.

• Stochasticity is handled explicitly. Because reasoning models often require positive sampling temperature, each model is given N = 5 attempts per task variation, and a variation counts as passed if solved correctly in the majority of attempts. The impact of stochasticity on systematicity is analyzed separately in Section 5.3.

• Not reported in the provided content: the specific model names evaluated, exact accuracy figures, dataset sizes, and the numerical outcomes of the reasoning-effort and sampling-noise analyses. Section 5 contains these, but the excerpt is truncated before the results.

Methodology in Plain English

The authors start from classic rule induction tasks in cognitive science — experiments where a learner sees a handful of examples and must infer a hidden rule. They wrap each task family in a systematic framework:

1. Build tasks with compositional structure. Each task family is procedurally generated, so an essentially infinite number of tasks of varying difficulty can be sampled. For example, an artificial-language family generates pseudolanguage commands (such as "dax fep" or "dax zup lug") that must be translated into sequences of colored shapes.

2. Generate structurally equivalent variants. Using task isomorphisms — the transformation τ = (π, σ, φ, ψ), where π and σ permute the support and query examples and φ and ψ are bijections remapping the input and output spaces — the authors create variants of a task that are guaranteed to be equally difficult in structure but different on the surface. In the Boolean example, the hidden rule "all red objects are wudsy" becomes "all blue objects are wudsy" with colors and shapes remapped accordingly.

3. Hold structure fixed, vary only the surface. Because isomorphisms are invertible and structure-preserving, any change in performance across variants cannot be attributed to a change in underlying difficulty.

4. Measure three things per task. For each task, K = 5 variations are sampled, and the authors record whether any variant was solved, what fraction was solved, and whether all were solved. The three metrics are monotonically decreasing, so they can be displayed as a single bullet chart.

5. Reduce noise. Each model gets N = 5 attempts per variant, with a majority-vote rule for deciding whether a variant was passed, to account for decoding variance in stochastic reasoning models.

6. Control for prior knowledge. Detailed system-prompt instructions describe the general structure of each task, limiting ambiguity and the influence of prior experience.

Why This Matters

Impact on research. The paper argues that a single-context success — a model solving one task — does not license a conclusion about the underlying ability. This connects directly to the language-of-thought hypothesis and to long-standing debates about whether neural systems exhibit systematic, compositional cognition. It also reframes systematicity evaluation as a specific type of metamorphic test: a check that a known relationship between related inputs holds in the outputs.

Real-world applications (extrapolated from the paper's framing, not separately listed in the paper):

• Benchmark design and model evaluation. The three-metric reporting scheme (any / fraction / all) offers a template for reporting robustness rather than a single headline accuracy.
• Reliability auditing of deployed reasoning systems. The finding that models can solve a task but not its isomorphic variants suggests that deployed systems may fail on inputs that are semantically equivalent to ones they handle correctly.
• Safety and alignment testing. The related-work section notes that adversarial brittleness remains a central concern, in particular for safety and alignment, and that models can still be derailed by subtle input changes.
• Cognitive-science comparison. Because the tasks come from established cognitive-science paradigms with human baselines, these evaluations can be placed alongside human performance rather than treated as purely synthetic benchmarks.

Industry relevance. Model providers increasingly expose reasoning models only with positive sampling temperature, which is why the paper builds stochasticity into its protocol with N = 5 attempts and majority voting. The finding that metamorphic invariance sometimes fails is directly relevant to anyone relying on a benchmark score as evidence of a general capability.

Future Directions

1. Extending isomorphic-variation testing to more task families and more transformation types, since the current study covers four families and four transformation types (symbol substitution, feature rebinding, constituent permutation, example reordering).

2. Determining why Boolean category learning is fully systematic while the other three families are not. The paper reports the contrast but the provided content does not report an explanation.

3. Clarifying the role of reasoning effort and sampling noise. Section 5.3 is described as studying the impact of stochasticity on systematicity, and Section 5 as investigating the impact of reasoning effort — both open the question of whether more thinking or different decoding settings close the systematicity gap.

4. Establishing whether systematicity failures reflect missing compositional structure or something else, given the paper's framing that systematicity can be understood as the behavioral outcome of successfully capturing a task family's underlying compositional structure.

Target Audience

Readers who will benefit most are researchers in NLP evaluation and benchmarking, cognitive scientists working on systematicity, compositionality, and the language-of-thought hypothesis, and engineers or policy-focused practitioners who need to know how much weight a reasoning model's benchmark success can bear. A background in machine learning evaluation is helpful; the formal definitions of tasks, isomorphisms, and metrics are stated precisely but illustrated with concrete examples, so the paper is accessible to readers without deep mathematical training. No prior familiarity with the specific cognitive-science paradigms (Boolean concept learning, Raven's progressive matrices, artificial grammar learning) is assumed.

Authors’ abstract

A central tenet of human cognition is systematicity, the principle that understanding one concept is inherently tied to understanding close variations of that concept. Do reasoning models robustly exhibit such systematicity? If so, we would expect consistent performance on structurally equivalent variants of the same task. Here, we extend established rule induction tasks from cognitive science to assess the systematicity of thought in current reasoning models. Each task family has compositional structure that we use to create structurally equivalent task variations through task isomorphisms such as recombination and substitution. We find that despite being able to correctly solve a task, models often fail on structurally equivalent variants of the same task. These findings suggest that many model behaviors lack systematicity, rendering it difficult to robustly establish the cognitive abilities of reasoning models beyond the particular contexts they were evaluated in.