BDH-CQ: In-Context Learning with Recurrent Latent Reasoning

AI summary

Overview

Research area: Neural sequence modeling and reasoning architectures, specifically in-context learning, latent (non-verbalized) reasoning, and evaluation on the Abstraction and Reasoning Corpus (ARC).

Technical level: Intermediate. The paper is readable without prior familiarity with the specific architecture, though concepts such as recurrent memory, latent states, and pass@k evaluation assume some machine-learning background.

Scope: The paper introduces BDH-CQ, a system that learns previously unseen visual transformations from demonstrations held in recurrent memory and solves queries through iterative computation in a continuous latent space, evaluated on ARC-AGI-1 with cost-efficiency and behavioral analyses.

What This Paper Is About

Most reasoning models either learn flexibly from examples in context but spend large numbers of tokens narrating intermediate steps, or reason compactly in latent space but require task-specific optimization before they can solve a new task. BDH-CQ aims to combine both: demonstrations presented at inference time update a recurrent memory, and the query is then solved by repeatedly transforming a continuous latent workspace without decoding intermediate reasoning into language. The goal is to test whether this combination can acquire and apply unseen visual transformations from context alone, at very low inference cost, on the ARC-AGI-1 benchmark.

Key Contributions

1. A combined architecture. BDH-CQ joins in-context learning through evolving recurrent memory with iterative reasoning in a structured continuous latent workspace, so that memory, adaptation, and inference form one computational fabric.

2. A new point on the ARC-AGI-1 cost–accuracy frontier. A 150M-parameter configuration reaches 29.5% pass@2 at a computed $0.00070 per task, which the authors state breaks through the previously reported Pareto frontier and establishes a new state of the art in benchmark cost efficiency.

3. A concept-organized capability and failure map. Using ConceptARC's 16 concept families, the authors separate isolated correct outputs from consistent rule application across the test inputs of a task.

4. Controlled ARC-like experiments. Post-freeze, oracle-generated tasks vary one source of complexity at a time to test what concepts the system learns from demonstrations and where it fails.

Main Findings

• ARC-AGI-1 result: The 150M-parameter system reaches 29.5% pass@2 in approximately 0.85 H200 GPU-seconds per task; at $3 per H200-hour, this is a computed cost of $0.00070 per task, described as less than one-tenth of a cent.

• Comparison to a hosted model: Using ARC Prize's reported costs as of July 2026, BDH-CQ is approximately 57x cheaper than GPT 5.6 Luna (Low), which scores 34.2% at $0.040. Accounting for OpenAI's 80% public API price reduction of GPT 5.6 Luna on July 30, 2026 (not reflected in ARC Prize's data as of August 6, 2026), BDH-CQ is approximately 11x cheaper.

• Headline benchmark numbers: On the 400-task public ARC-AGI-1 evaluation set, pass@1 is 97 tasks (24.25%) and pass@2 is 118 tasks (29.50%), with a descriptive Wilson 95% interval of [25.24, 34.15]. Test-pair results are 108/419 (25.78%) at pass@1 and 130/419 (31.03%) at pass@2.

• ConceptARC, semantic identifiers: 95/160 tasks (59.38%) pass@2 and 73/160 (45.63%) pass@1; test pairs 374/480 (77.92%) pass@2 and 332/480 (69.17%) pass@1.

• ConceptARC, opaque identifiers: 96/160 tasks (60.00%) pass@2 and 72/160 (45.00%) pass@1; test pairs 374/480 (77.92%) pass@2 and 334/480 (69.58%) pass@1.

• Concept families vary widely: In the opaque replication, ExtendToBoundary, FilledNotFilled, and TopBottom2D each reach 9/10 pass@2, while Copy and Order reach 2/10. With ten tasks per family, the authors note the descriptive Wilson intervals for 9/10 and 2/10 overlap broadly (59.6%–98.2% and 5.7%–51.0%).

• Consistency gap: There is an 18.5-point gap between semantic ConceptARC pair accuracy (77.92%) and strict task accuracy (59.38%). Under pass@2, the system solved zero test pairs for 13 tasks, one for 15 tasks, two for 37 tasks, and all three for 95 tasks, meaning 52/160 tasks have one or two correct inputs but are not solved as tasks. Copy reaches 19/30 semantic test-pair pass@2 but only 2/10 strict tasks.

• Extrapolation varies by operation: Propagation remains correct on 48/48 held-out outputs at both pass@1 and pass@2 across distances 2–8, and copying remains correct on 48/48 as target sites increase from one to four. Ordering is nearly saturated through five objects, then falls to 29/36 outputs at length six, 8/24 at seven, and 1/24 at eight at pass@2. Nesting stays nearly saturated through depth four but falls to 29/36 at depth five.

• Two different failure signatures: At ordering length eight, only 3/24 held-out outputs have the correct dimensions. At depth-five nesting, all 36 outputs have correct dimensions and mean best-candidate cell accuracy exceeds 99.9%, with errors typically differing from the target in a single containment decision.

• Demonstration coverage helps: Re-running byte-identical inputs under a "supported" context (one demonstration at the test complexity) raises depth-five nesting from 19/24 to 24/24 exact outputs at pass@2; for ordering it recovers 13/24 outputs from a 0/24 baseline. Ordering goes from 0/24 to 12/24 at pass@1 and 0/24 to 13/24 at pass@2; nesting goes from 15/24 to 16/24 at pass@1 and 19/24 to 24/24 at pass@2.

• Strong contextual binding: On a task where demonstrations define a fresh color permutation, the system solves all 96 held-out outputs at rank one as the number of simultaneous bindings increases from two to eight (24/24 at every level).

• Composition is operation-dependent: Across three dense 3×3 motif families with 72 held-out outputs per condition, relocation alone, atomic reflection, and atomic rotation are each solved on 72/72. Rotation composed with relocation is 72/72; reflection composed with relocation is 47/72. Color swap is acquired atomically only in the original family (26/72 pooled) and is never composed with relocation (0/72); the two shuffled families each reach only 1/24 in isolation.

• Identifier and batch context do not explain the score: The opaque replication replaced semantic task IDs with cryptographically opaque labels and mixed concept areas within batches. Aggregate performance was unchanged: 374/480 test pairs in both conditions and 96/160 opaque versus 95/160 semantic tasks, with exactly six semantic-only and six opaque-only pass@2 successes. Outputs were not identical, however: 442/480 first candidates, 276/480 complete ordered candidate lists, and 455/480 attempt counts agreed.

• Rank-1 delivery: In the opaque ConceptARC replication, all 75 single-candidate records were already correct at rank one, so nominal pass@2 should not be read as two independent sampled attempts for every input.

• Effort scaling: Training across latent reasoning effort levels yields HIGH 29.5% pass@2 with 0% cost reduction, MEDIUM 27% with 11% cost reduction, and LOW 21% with 22% cost reduction.

• MIN versus STANDARD effort: MIN cost $0.00088399 versus $0.00265246 per task and scored 111/400 rather than 118/400 pass@2, a difference of −1.75 percentage points. The paired split was 105 tasks solved by both, 13 standard-only, 6 min-only, and 276 neither (two-sided exact McNemar p = 0.167), which the authors call statistically unresolved in favor of standard.

• Repeatability: Repeated identical requests were byte-identical at both effort tiers; the standard re-run matched all 419 ARC-AGI-1 test inputs from the earlier standard run.

• Surface properties explain little: Across 39 bucketings of grid size, color count, object count, and demonstration count on the public set and 38 on the calibrated generated set, nine public-set bucketings exceed the permutation search null and none do on the generated set. Grid size is the strongest individual descriptor but remains weak, with pseudo-R² = 0.072 on public tasks and 0.010 on generated tasks.

• Mechanic stratification: On a mechanic-stratified generated set of 1,131 tasks (337 solved, 29.8%), solve rates span 65.7 percentage points, from 68.6% for flood fill (n=51) to 2.9% for gravity and stacking (n=68). Other mechanics include denoising 56.9% (n=65), scaling 53.7% (n=82), cropping and extraction 44.6% (n=74), translation 35.6% (n=87), tiling and repetition 34.2% (n=79), line drawing 29.8% (n=84), rotation and recolor by property 25.3% each (n=75 each), object sorting and rank 19.1% (n=68), object counting 18.6% (n=86), reflection 16.7% (n=72), symmetry completion 16.4% (n=61), occlusion repair 16.1% (n=56), and panel set operation 10.4% (n=48).

• Cohort comparison: Exact whole-task pass@2 is 29.5% (118/400) on the public ARC-AGI-1 evaluation, 37.2% (149/400) on a calibrated generated set, and 29.8% (337/1,131) on the mechanic-stratified generated set; the authors note the generated cohorts serve different purposes and are not directly comparable.

Methodology in Plain English

The system is built on the Dragon Hatchling (BDH) architecture family, which uses high-dimensional positive activations, low-rank communication, and a recurrent associative state. Two components do different jobs. A recurrent memory processes the demonstration pairs one at a time and accumulates task information, with fixed parameters and no explicit key–value cache that grows with context. After all demonstrations are ingested, a separate latent workspace is initialized from the query and then repeatedly transformed for a fixed number of iterations before the answer is decoded. Intermediate reasoning states are never turned into words.

For training, the authors built a curated ARC-style dataset combining privately curated examples with the public ARC-AGI-1 training set, RE-ARC, ConceptARC, ARC-Heavy, and ARC-GEN100K, plus additional augmentations. Neither task identifiers nor evaluation-task demonstration pairs participate in training, and no parameters are updated at inference time. Dimensions, exact update rules, and the internal training recipe remain proprietary.

Evaluation uses the 400-task public ARC-AGI-1 evaluation split under the leaderboard's two-attempt (pass@2) convention, producing up to two ranked candidates. The reported dollar value comes from measured hardware time. An independent black-box audit by co-authors from Bielik and New York University reproduced the 29.5% pass@2 score without access to model weights.

For behavioral analysis, the authors combine the ConceptARC ontology (16 families, ten tasks and thirty test inputs each, with strict task accuracy requiring all three test inputs to be solved), fresh ARC-like tasks generated after freezing the model with deterministic oracle outputs, and replications that replace semantic identifiers with opaque labels and mix concept areas within batches.

Why This Matters

The paper's central claim is that in-context learning and latent reasoning can be combined in a compact system rather than developed separately, and that this combination can be extremely cheap per task. If the reported cost-efficiency holds up, it changes the economics of what inference-time reasoning can be afforded and provides a diagnostic map of which visual concepts transfer and which do not.

Impact on research: The work proposes that demonstration-driven adaptation and non-verbalized iterative computation can occupy the same architecture, and it offers a concept-organized profile of capabilities rather than a single aggregate score. The controlled experiments distinguish failures of extrapolation from failures of execution — matched demonstrations eliminate the depth-five nesting failures but leave an ordering bottleneck.

Real-world applications:

• Low-cost visual puzzle and abstraction solving where per-task inference budget is a constraint.
• Programs that must infer a transformation rule from a handful of examples and apply it exactly, such as spreadsheet-style or diagram-to-diagram transformations of gridded data.
• Constraint-satisfaction tasks that benefit from iterative latent refinement, following the authors' earlier Sudoku work.
• Deployments where reasoning must happen without emitting intermediate natural-language traces, for latency, bandwidth, or disclosure reasons.

Industry relevance: The reported operating point of $0.00070 per task, and the comparison of approximately 57x cheaper than GPT 5.6 Luna (Low) (or approximately 11x after the July 30, 2026 price reduction), speaks directly to production cost planning. The paper also states that the architecture admits tensor sharding patterns inherited from BDH that make it particularly easy to train at 1T scale, and that early experiments confirm Transformer-like scaling laws during pretraining from 1B to 600B parameters while preserving latent reasoning capabilities.

Future Directions

1. Scaling BDH-CQ itself. The authors note the current result comes from a compact model that leaves a large cost budget for increasing capacity, and they plan larger models and longer training runs to test whether boundaries move predictably with scale.

2. ARC-AGI-2 and breadth of domains. ARC-AGI-2 is named as the next visual-reasoning target, with a development agenda around output construction, conditional binding, demonstration coverage, and multi-operator composition. Sudoku and other constraint-satisfaction domains are proposed as complementary tests of long-horizon latent refinement.

3. Language and mathematical reasoning. Future work would test whether the same recurrent memory can acquire tasks from textual demonstrations while retaining BDH's sequence-modeling capabilities.

4. Combining continuous and verbalized reasoning. Because BDH layers support language modeling as well as latent reasoning, the authors propose systems that combine continuous internal computation with decoded intermediate language when communication, verification, or tool use benefits from it.

Open questions the paper leaves unresolved include why color swapping is acquired only in the original fixed-layout motif family but never composed with relocation, whether the statistically unresolved MIN-versus-STANDARD effort comparison would resolve with more data or different default effort, and what full factorially separated experiments on identifiers and batch composition would show — the combined intervention in this paper does not isolate those two effects.

Target Audience

Researchers working on reasoning architectures, in-context learning, and latent or continuous-thought computation; practitioners evaluating ARC-AGI-1 under cost constraints; teams interested in compact models that reason without verbalizing intermediate steps; and readers who want a concept-level diagnostic of where such a system succeeds and fails rather than a single benchmark number. Readers looking for full architectural details, exact update rules, or the complete training recipe will not find them here, as the authors state these remain proprietary.

Authors’ abstract

We introduce BDH-CQ, a reasoning model that combines in-context learning with recurrent latent reasoning. Inputs presented at inference time continuously update the model's recurrent memory; the model then solves a query through iterative computation in a high-dimensional latent space, without verbalizing its intermediate reasoning. We evaluate the model on the public ARC-AGI-1 evaluation set and use controlled ARC-like interventions to study what it learns from demonstrations, how consistently it applies an inferred transformation, and which concepts remain difficult. A 150M-parameter configuration reaches 29.5% pass@2 at a computed inference cost of \$0.0007 per task. This operating point breaks through the previously reported ARC-AGI-1 cost-accuracy Pareto frontier, establishing a new state of the art in benchmark cost efficiency.