The Pulse
Huginn’s Sandwich Design Beats Ouro in Matched Model Tests
A study submitted to arXiv on October 16 compares Huginn’s prelude-loop-coda design with Ouro’s fully recurrent Transformer stack. At matched scale, Huginn performs better on reasoning and context tasks, though input injection and other des
AI.info Team · Written with AI · fact-checked ·
A Huginn-style Transformer outperformed Ouro across all 10 benchmarks in a matched 730-million-parameter study, but the advantage was not uniform across tasks. The researchers say the strongest gains came from placing repeated computation in the middle of the network; repeatedly feeding the prompt back into that loop helped with context-sensitive tasks but could hurt mathematical reasoning.
The study, “Towards Looped Models Done Right, Part I,” was submitted to arXiv on October 16, 2026. Its seven authors compare models trained from scratch at matched parameter scale, logical depth and token budgets, then test whether the design findings carry over to mixture-of-experts models.
Huginn moves the loop to the middle
Looped language models reuse Transformer blocks across multiple computational steps, reducing the number of distinct weights that must be stored while allowing a model to perform more logical layers of computation. Ouro’s design loops the full Transformer stack. Huginn instead places a repeated core between a one-time prelude, which processes the input, and a coda, which produces the output.
In the dense comparison, the study matched both designs at 730 million stored parameters and 112 logical block executions. Ouro repeated its 28-block stack four times. Huginn used an eight-layer prelude, an eight-step loop through a tied 12-layer core, and an eight-layer coda. The researchers evaluated both against 10 tests covering knowledge, reasoning, mathematics and code generation.
“The prelude-recurrent block-coda sandwich architecture consistently improves performance on context extraction and reasoning-intensive tasks,” write Benhao Huang and his co-authors in the paper’s abstract. Benhao Huang et al., “Towards Looped Models Done Right, Part I”
Reasoning gains came with uneven trade-offs
The sandwich arrangement raised Huginn’s MATH500 score by 12 points and its DROP score by 2.61 points compared with the full-stack Ouro control. The researchers report that gains on MATH500 and BBH-CoT persisted across four training budgets. Knowledge-focused tasks and tasks with strict output requirements, however, showed no consistent improvement, and some scores fell.
The study also tested input injection: adding the prelude’s representation of the prompt to the recurrent state before each loop. For Huginn, that change improved scores on MMLU, BBH-CoT, DROP and two code-generation tests. It cut MATH500 by 3.60 points and GSM8K by 2.51 points, a reminder that making prompt information easier to retrieve does not necessarily aid every kind of computation.
Other architectural choices produced less decisive results. Randomly initializing the recurrent state helped on some benchmarks and hurt on others; a separate high-and-low recurrent-state design showed no consistent benefit. The authors say the tests do not establish how random initialization affects path independence or performance at loop depths beyond those used in training.
The advantage carried into sparse models
For the mixture-of-experts comparison, both looped models had 8.0 billion resident parameters and about 794 million active parameters per physical pass. After 500 billion training tokens, Huginn scored higher than Ouro on eight of 10 benchmarks. Its largest reported leads were 4.70 points on GSM8K and 3.60 on MATH500; Ouro held smaller leads on ARC-Challenge and BBH-CoT.
Huginn also recorded a lower expert load-balancing loss than Ouro, which the authors interpret as more even use of the model’s experts. Against a separate 112-layer feedforward MoE baseline, though, Huginn did not win overall: that baseline led on seven of 10 tests, while Huginn beat it on DROP and GSM8K and matched it on MATH500. The paper says Huginn used 75% fewer resident parameters than that feedforward reference under matched training and inference FLOPs.
Code and checkpoints accompany the work
The findings suggest a design preference, not a universal win for every Huginn component. Loop placement produced the clearest gains in the dense tests, while input injection shifted performance depending on whether a task relied on prompt context or multi-step quantitative reasoning. The researchers call for further controlled comparisons as looped-model designs develop.
The authors’ xLLM-Loop repository provides training and evaluation code and lists released checkpoints for the work. The dense comparison’s central result remains specific: with 730 million stored parameters and 336 billion training tokens, the complete Huginn design beat Ouro on all 10 tested benchmarks.