The Pulse
Dream-RSI Cuts AI Discovery Costs by Replaying Agent History
An arXiv paper introduces Dream-RSI, a system that uses historical discovery trees as replay simulators to test and improve exploration policies before deploying them in costly online searches.
AI.info Team ·
Dream-RSI uses history to reduce discovery costs
A new research system reports matching or improving several AI discovery results while using fewer agent calls than existing methods. Dream-RSI treats completed discovery histories as replay simulators, allowing alternative exploration policies to be tested without repeatedly generating code or rerunning evaluators.
The work, presented in a 12-page arXiv paper submitted on September 14, 2026, targets a weakness in autonomous discovery systems: exploration strategies can remain fixed even after an agent has accumulated evidence about which search paths work and which do not.
“Our key insight is that accumulated discovery history can serve as a replay simulator over the realized search space.”
Tong Zheng, first author of the paper
Dream-RSI, short for recursive self-improvement through evolving worlds, stores discovery histories as trees containing branches, generated artifacts, evaluation results, diagnostics, and scores. Candidate exploration policies can move through those recorded trees without generating new code or rerunning the evaluator.
History becomes a test bed for new search policies
The method separates discovery into online and offline stages. During online exploration, a coding agent follows an executable policy that decides which branches to extend, how many attempts to run in parallel, and when to stop. Each completed rollout adds a new discovery tree to the system’s history.
During offline “dreaming,” a policy-development agent revises the exploration code and tests each version against the stored trees. Replay reveals previously recorded outcomes deterministically, allowing the system to compare alternative branch choices, exploration orders, batch sizes, and stopping decisions without paying for another full discovery run.
The paper’s replay score combines three elements: the best solution found, a penalty for the number of represented attempts, and a bonus for completing more attempts per decision round. Dream-RSI selects the best-performing policy version on the accumulated history, deploys it in a new online rollout, and adds that rollout to the replay pool.
Lower cost on Lasso solver discovery
The clearest result comes from an algorithm-engineering task focused on discovering an efficient Lasso regularization-path solver. The researchers evaluate the resulting programs on six held-out datasets covering biological and non-biological workloads, then compare Dream-RSI with standard libraries, SimpleTES, and a fixed exploration strategy.
Using Gemini-3.1 Pro as the discovery agent, Dream-RSI produces an average runtime of 2,931.0 milliseconds across the six datasets with 317 discovery-agent calls. Recursive Fixed Exploration records an average runtime of 3,587.1 milliseconds with 550 calls.
A second experiment using Gemini-3.7 Flash records a smaller but still measurable gain. Dream-RSI reaches an average runtime of 2,350.6 milliseconds with 1,879 calls, compared with 2,516.7 milliseconds and 3,200 calls for fixed exploration. The paper says both discovered solvers outperform the standard sklearn and glmnet implementations on all six held-out datasets.
The reported savings are measured in discovery-agent calls, not as a claim about total electricity use, dollar cost, or wall-clock time across every component of the system. Against SimpleTES, which uses 51,200 generations in the cited comparison, Dream-RSI achieves lower average downstream runtime with roughly two orders of magnitude fewer discovery-agent calls.
Results span mathematics and GPU kernels
The researchers test Dream-RSI on eight tasks across algorithm engineering, mathematical optimization, and GPU kernel engineering. In mathematical optimization, the system matches or surpasses strong baselines within 1,000 generations, yielding more than 50 times the budget savings compared with SimpleTES.
GPU kernel experiments use four KernelBench tasks: VGG16, LayerNorm, ConvDiv, and ConvMax. Dream-RSI reaches target execution speeds using 2.43 times fewer generations on VGG16 and 1.79 times fewer on LayerNorm. Under identical budget constraints, it reports up to 2.09 times higher performance on ConvDiv and up to 1.44 times higher performance on ConvMax.
The system uses Gemini-3.1 Pro for the kernel experiments and keeps the coding agent, evaluator, initialization, and resource limits aligned with the fixed-exploration comparison. Only the exploration-policy code changes between recursive rounds.
Replay improves search without forcing a single direction
The paper also compares interactive replay with a simpler approach that summarizes prior runs into high-level directional advice and inserts that advice into later prompts. On the ConvDiv kernel task, the explicit guidance performs worse than the replay-based method under equivalent budgets.
The authors attribute the difference to the way broad instructions can narrow a search too early. Replay lets a policy inspect concrete branches and recorded outcomes while retaining the option to open different paths, adjust parallelism, or stop unproductive work. In one analysis, the evolving policy initially reduces the number of evaluated attempts from 110 to 50 as performance improves, then increases exploration again after progress levels off.
A narrow claim with wider significance
Dream-RSI does not change the underlying coding agent, evaluator, or execution interface. It changes the controller that allocates discovery effort, and its contribution is to make that controller testable against history before another costly online rollout.
The reported results are experiments from a single arXiv submission rather than evidence that the approach will transfer unchanged to every autonomous agent or scientific search problem. They show an alternative to treating agent history as passive context: store the branching structure and outcomes, then use the record to evaluate new search behavior before spending more calls.
The paper is available from arXiv, with linked project materials for Dream-RSI.