SanSi: A Looped Typed Decision Model for System 1.5 Thinking

AI summary

Overview

Research area: Natural Language Processing — typed decision models, looped/recurrent transformers, latent reasoning, and inference-time computation.

Technical level: Intermediate. The paper assumes familiarity with transformer layers, LoRA adapters, proper scoring rules, and reinforcement learning from model-based rewards, but its core argument (apply the same layers several times instead of adding parameters) is explainable without that background.

Scope: The paper asks whether recursively re-applying the same 24 transformer layers before a single typed probability readout — "System 1.5" — buys accuracy, reasoning depth, and better-calibrated probabilities that a single forward pass does not.

What This Paper Is About

Many language model calls do not ask for text but for a decision: the caller declares a set of options (for example, spam and not spam) and the model returns one probability per option in a single forward pass, with no generated text to parse. A single pass gives a fixed amount of computation to every decision, which is enough for a message-level judgment but not for decisions that require following a chain of dependent steps. The authors study a middle path between one pass (System 1) and generated reasoning (System 2): looping, in which the same layers are applied recursively to their own output so the model can revise its hidden state before committing, without generating any token.

Key Contributions

1. SanSi, a looped typed decision model. Built on Ouro-1.4B and Ouro-2.6B, language models pre-trained to loop, SanSi keeps the backbone frozen and trains only LoRA adapters of rank 64 on all attention and feed-forward projections (60.6M parameters, shared across loops) plus per-loop readouts (0.27M), for 61M trainable parameters in total at the 1.4B size.
2. Per-loop supervision with a proper scoring rule. Option probabilities are read after every loop and each read is trained with the sum of cross-entropy and the Brier score, backpropagated through all preceding loops, so one model serves every budget from one loop to eight in a single run.
3. A controlled measurement of what looping adds. Across 10,027 test decisions from 59 sources, with identical data, recipe and seeds, looping is compared against a non-looped model of the same shape and against newer single-pass models of equal and triple the parameter count.
4. Depth-controlled and use-case evaluations. Two program-generated tasks (liar chains, object swaps) test whether loops solve depths never seen in training, and a reinforcement learning study tests SanSi as the sole reward for training a generator without gold answers.

Main Findings

• Looping adds 13.5 points at fixed shape, data and recipe. SanSi reaches 72.0% accuracy against 58.4% for SmolLM2-1.7B, the closest non-looped model (Ouro's tokenizer, 24 layers, hidden size 2,048), trained on the same items with the same readout. The gain holds in every test group and on 55 of 59 test sources, and is stable across three seeds (12.9–14.8 points).
• The backbone is not the cause. Ouro-1.4B trained and run with a single loop reaches 58.6%, indistinguishable from SmolLM2 (+0.1 [−0.7, 1.0]), so the 13.5 points come from the loops rather than from the pre-trained backbone.
• Gains are largest where a single pass is insufficient. Only +3.1 points on classification, but +15.1 on multi-step reasoning, +16.7 on long documents, +14.2 on sentence pairs, and +17.6 on knowledge questions, which occur only in the test set. By distance from training: +10.5 in-distribution, +10.0 near transfer, +15.8 far transfer.
• Per-parameter efficiency beats a newer model, per-computation it does not. SanSi draws level with Qwen3.5-2B (1.9B parameters) at loop 2 and leads it by 5.3 points at loop 8. Qwen3.5-4B (4.2B parameters) stays 1.8 points ahead at loop 8, where SanSi uses 3.3 times its GPU time, and 3.4 points ahead at equal computation (three loops).
• The single-pass reference does not depend on the authors' recipe. Trained on the same data with Kev's own code, Qwen3.5-4B reaches 74.3%, 0.5 points above Qwen3.5-4B with the authors' recipe, and 2.4 points above SanSi with a third of its parameters. SanSi-2.6B, the same recipe on Ouro-2.6B, reaches 75.8% and is 1.5 points ahead of Kev-4B with 63% of its parameters.
• Cost is paid in computation, not parameters. Inference takes 7.7 times and training 9.5 times the GPU time of the one-loop model. Three loops cost three eighths of the full pass and already reach 70.4%.
• The gain arrives early. Loop 1 matches the one-loop model (58.4%, −0.2 [−0.6, 0.3]), loop 2 adds 8.5 points, loop 3 adds 3.5, loop 4 adds 1.2, and loops 5–8 together add 0.4. Three loops deliver 88% of the gain from loop 1 to loop 8; the second loop brings 62% of that gain and the third 88%.
• Fixes outweigh breaks only until loop 5. On the 9,645 answerable items, loop 8 fixes 21.0% of loop-1 answers and breaks 7.3%. After loop 4, 90.2% of answers are final and remaining fixes (4.0%) barely exceed breaks (3.6%); after loop 5 they are equal. At most 11.7% of items are right at some loop but wrong at loop 8.
• Harder items settle later, and late answers are unreliable. 56.7% of answers never change after loop 1 and 89.1% have settled by loop 4. Items solved by all three single-pass models settle after 1.4 loops on average, by two after 2.4, by one after 3.0. 82.2% of never-changing answers are right, but only 35.6% of those settling at loop 8, with mean confidence falling from 0.89 to 0.44.
• Confidence rises on wrong answers too. Mean confidence rises from 0.769 to 0.869 on right answers and from 0.567 to 0.660 on wrong ones, with the gap near 0.20 throughout; the AUROC separating right from wrong improves only from 0.760 to 0.795.
• Calibration peaks at loop 3. ECE falls from 0.105 to 0.082 at loop 3, then climbs back to 0.093 at loop 8 as confidence keeps rising after answers settle.
• Loops detect missing evidence. On the 382 items whose key evidence was removed, hard answers drop from 27.1% at loop 1 to 17.5% at loop 8, and evidence AUROC rises from 0.835 to 0.935. Only Qwen3.5-4B (0.948) and Kev-4B (0.947), both three times the parameters, score higher.
• Loops extend solvable depth beyond the training range. Trained on liar chains up to k ≤ 8 and tested to k ≤ 16, SanSi holds depth 3 after one loop, 6 after two and 11 after four. On the unseen depths 9–16, one loop is at chance (49.9%) and eight loops reach 72.6%, against 50.0% (chance) for Qwen3.5-4B. SanSi leads Qwen3.5-4B by 22.7 points on unseen liar-chain depths and 22.6 on trained ones, and by 20.2 and 29.5 on object swaps.
• Running past the trained loops hurts. SanSi falls from 72.0% at loop 8 to 70.1% at loop 16; the trained loops cap its compute.
• Averaging the loops is a free calibration fix. Averaging the eight loops' option probabilities keeps loop-8 accuracy (71.8% against 72.0%) and halves ECE (0.044 against 0.093), needing no labels or training.
• Looping helps as a reward signal. As the only reward for GRPO training of SmolLM2-1.7B on 2WikiMultiHopQA without gold answers, SanSi read at loop 8 raises token F1 from 39.5 to 47.3; loop 4 gives 45.8; loop 2 leaves F1 unchanged; loop 1 damages the generator (29.1, −10.5). In the first 20 training steps, the loop-1 reward separates gold-matching answers worse than loop 4 or loop 8 (AUROC 0.78 against 0.91 and 0.90).

Methodology in Plain English

The authors start from Ouro, a language model whose entire 24-layer stack was pre-trained to be applied repeatedly, and freeze it. A decision item is rendered as a prompt listing the options under letters A, B, … and ending in "Answer:". After each loop, they take the hidden state at the final prompt token and produce a probability distribution over the declared options using the frozen language-model head for those letters, plus a small rank-16 correction and a per-loop scale that start at the identity.

Training supervises every loop, not just the last. Each loop's distribution is scored against the target with cross-entropy plus the Brier score, and the loss of every loop is backpropagated through all the loops before it. Targets are one-hot when there is a correct option, uniform for items whose state lacks the needed evidence, and the annotators' label distribution for crowd-labelled items. Because every loop is read and trained, a single trained model can be evaluated at any budget from one to eight loops, and the authors read it at each loop in their analyses.

The test suite combines the Kev training and transfer suites with the public JevBench items, giving 12,800 training items from 20 sources and 10,027 test items from 59 sources, split into six item types and four groups by distance from training (in-distribution, near transfer, far transfer, and the 231 public JevBench items). Comparisons use four single-pass models trained with the same recipe, the same items in the same order, the same LoRA rank, the same readout and the same loss, with three seeds each; a fifth single-pass model, Kev-4B, checks that the reference holds under Kev's own recipe. Accuracy, expected calibration error, evidence AUROC and GPU-time cost are reported, with 95% bootstrap intervals over groups of related items.

Why This Matters

Impact on research. The paper isolates looping as a distinct axis of inference-time computation for decision tasks, separate from adding parameters and separate from generating reasoning tokens. It shows that a fixed share of the accuracy gap between a 1.4B and a 4.2B model can be closed without new parameters, that loops generalize to reasoning depths never seen in training where a larger single-pass model is at chance, and that the probabilities callers depend on improve in some respects (missing-evidence detection) but degrade in others (calibration after loop 3).

Real-world applications.

• Content moderation and abuse triage, where a decision must be accepted, escalated or routed on the basis of its probability.
• Policy and compliance checks over contracts or rules, where the decisive evidence may be absent and the model should decline to give a hard answer.
• Multi-hop question answering over documents, where chains of relations must be followed before an answer is committed.
• Model-as-judge reward signals for training generators in settings where no gold answer exists.

Industry relevance. The typed-decision contract (declare options, get probabilities, get no text) is already commercialized through the Jev API and open reimplementations such as Kev. The paper is directly relevant to anyone choosing between a larger single-pass model and a smaller looped one: it reports that per parameter the looped model wins, and per unit of computation the larger one wins, so the trade can be set at test time by choosing how many loops to run.

Future Directions

• A rule that chooses the loop budget per item: at most 11.7% of items are right at some loop but wrong at loop 8, which bounds any such selection rule, and the authors leave early stopping on easy items to future work.
• Extending the findings beyond the Ouro family: the gain rests on looped pre-training, and how far it carries to other and larger looped backbones is stated as an open question.
• Improving calibration after answers settle, since confidence keeps rising after loop 3 even though accuracy is flat, and averaging the loops reduces ECE without fixing the underlying behavior.
• Broadening the verifier case study, which uses one dataset (2WikiMultiHopQA) and one generator and is described as an illustration rather than a general evaluation of SanSi as a reward model.
• Testing the suite beyond English and separating any contamination, since the authors cannot exclude that a backbone has seen some of the public datasets during pre-training.

Target Audience

Researchers and engineers working on efficient inference, latent or recurrent reasoning, and decision-type model interfaces; practitioners comparing parameter scaling against inference-time computation for classification, policy checks, or model-as-judge reward signals; and readers interested in calibration and uncertainty for models whose probabilities are used to accept, escalate or route decisions.

Authors’ abstract

Typed decision models answer a declared question without generating text: a decision head returns a probability for each of the declared options in a single forward pass. A single pass is fast, intuitive System 1 thinking. We study what lies between one pass and generated reasoning: looping, in which the same layers are recursively applied several times before one typed readout. Each loop lets the model revise its hidden state before it commits to an answer, without generating a token; we call this System 1.5 thinking. We propose SanSi, which turns a pre-trained looped language model into a typed decision model. The option probabilities are read after every loop, and every loop is trained with a proper scoring rule, so that one model serves every budget from one loop to eight in a single run. On 10,027 test decisions from 59 sources, SanSi reaches 72.0% accuracy: 13.5 points above a non-looped model of the same shape trained with the same recipe, 5.3 points above a newer non-looped model of its size, and 1.8 points below one with three times the parameters. On two depth-controlled tasks, loops extend the solvable depth beyond the depths seen in training, where the larger single-pass model fails. Used as the judge for policy optimization with reinforcement learning, without gold answers, SanSi raises the generator's F1 by 7.7 points.