Evolving with Open Worlds: Towards Recursive Self-Improvement for Future Forecasting

AI summary

Overview

Research area: Natural Language Processing / AI agents — specifically recursive self-improvement (RSI), agentic forecasting of real-world events, and reinforcement learning with verifiable rewards.

Technical level: Intermediate. The paper is readable without deep mathematical background, but it assumes familiarity with LLM agents, tool-use/search pipelines, and fine-tuning paradigms such as reinforcement learning with verifiable rewards (RLVR) and rejection fine-tuning (RFT).

Scope (one sentence): The paper proposes DeepEvolve, a three-module system (a live task-generating environment, an agentic forecaster, and a recursive-evolution engine) that lets a forecasting agent improve from its own resolved predictions, and evaluates it on the weekly FutureX leaderboard and on a self-synthesized live evaluation.

What This Paper Is About

Nearly all deployed AI systems ship with frozen parameters and only get better through external human effort — new curated data, new offline training runs. The paper asks what it would take for a system to keep improving from its own experience instead, a property the authors call recursive self-improvement. The authors argue this cannot be studied on fixed datasets, because a static benchmark neither renews the challenge nor judges progress on its own, so they build the whole study around real-world event forecasting: questions whose answers do not yet exist, resolve on their own with time, and are scored against what actually happened.

Key Contributions

1. A live, verifiable, non-stationary environment for studying self-improvement. The authors construct Prometheus, which ingests daily news from 125 websites and synthesizes forecasting questions about events resolving the following day, then validates them through temporal checks, deduplication, conflict detection, and diversity-aware selection. Ground truth arrives on its own from the world rather than from an answer key.

2. A function-oriented agent design that makes live experience trainable. Branchus splits forecasting into a time-sensitive search team (scheduler, searcher, critic) that gathers evidence while a question is still open, and a time-invariant predict team that reasons only over cached evidence and trajectories. This decoupling lets many RFT/RLVR rounds read from a frozen pool instead of a web that has already revealed the answer.

3. A recursive-evolution engine (Athena) that turns one trajectory into three kinds of capability. The same prediction trajectory is (a) reinforced into the predict team's parameters against its resolved outcome, (b) distilled into reusable, inspectable forecasting skills inserted at inference without touching weights, and (c) mined for failure modes that become task-synthesis skills, which Prometheus uses to generate harder questions for the next round.

4. Evidence that the three levers compose into compounding gains. The paper isolates and compares parameter evolution, skill evolution, and environment evolution, showing where each helps, what each costs, and why the loop as a whole outperforms any single lever.

Main Findings

• Top rank on the external FutureX leaderboard. FutureX evaluates predictions against real-world outcomes and is updated weekly; the self-improved forecaster ranked first across all systems over three consecutive weeks, with up to a 195% improvement in rating.

• Top-three across four base LLMs on the synthesized evaluation. Against a baseline suite of ReAct, MAS, OpenClaw, DeerFlow, and MiroFlow, the four Branchus configurations occupy the top three positions in a single ordering over all 23 harness–LLM systems. Branchus with Qwen3.8-Max reaches a Bradley–Terry score of 62.2 and rank 1; GPT-5.6-Sol reaches 57.9 (rank 2); DeepSeek-V4-Pro reaches 56.4 (rank 3); Claude-Opus-4.6 reaches 53.1 (rank 9).

• The margin over the strongest baseline never changes sign. It ranges from +10.7 to +0.5 Bradley–Terry points depending on the base model.

• Calibration, not raw accuracy, is where the advantage lives. Baselines occasionally edge out Branchus on choice accuracy by a few tenths of a point (for example MAS on DeepSeek-V4-Pro and OpenClaw on Claude-Opus-4.6). Branchus instead attains the best Brier score in three of four blocks and the best relative WIS on Qwen3.8-Max (0.737) and GPT-5.6-Sol (0.865), meaning tighter, better-calibrated intervals rather than over-confident guessing.

• Model strength and agent design are complementary. A stronger LLM lifts every system, but Branchus on the weaker DeepSeek-V4-Pro still reaches rank 3 and outranks every baseline regardless of the model those baselines use.

• RLVR sharpens probabilistic judgment more than top-1 answers. Across two LLMs, choice accuracy rises by 1.3 points each, while the numeric score (1 − bWIS) improves by 0.31 and 0.38. The authors read this as calibration being the more sensitive signal.

• RFT gains compound across rounds. Accuracy rises 0.7 to 2.9 points. Training GLM-4.7-Flash on round-one cached trajectories improves Brier from 0.651 to 0.636 and relative WIS from 1.02 to 0.99; a second round with a pool grown from 600 to 1,000 examples pushes Brier to 0.630 and relative WIS to 0.98.

• A handful of live trajectories is enough for skills to help. Distilled skills drop the Brier score from 0.135 to 0.122 and MAPE from 2.80% to 2.28%, with no weight updates at all.

• Skill gains accrue round by round but with diminishing aggregate effect. Over nested pools of 88, 186, and 262 trajectories, Brier lands 0.007, 0.012, and 0.013 below the skill-free baseline, and MAPE descends from 2.80% to 2.40%, 2.32%, and 2.28%. Later rounds buy breadth rather than headline score: topical coverage grows from 168 domains and 173 tasks at round 1 to 479 domains and 505 tasks at round 3, approximately threefold.

• Skills and parameters act at different stages. On structure synthesis problems with recurring patterns, skills cut tool calls by 44% (from 32 to 18 per question) while MAPE stays at least as strong as the skill-free baseline. On open synthesis problems from unfamiliar domains, tool calls rise 13% (from 30 to 34) and MAPE falls 32% (from 3.40% to 2.31%). Parameter updates, which touch only the predict team, change nothing about search behaviour — they change how well the same evidence is used.

• Externalization is cheap but capped; internalization is costly but goes further. Skill extraction is pure inference with no rollout and no backward pass, a gap that widens with model size; one trajectory costs 14.96 calls and 303,135 tokens in total across evaluation, extraction, and generalization stages. But roughly 88 trajectories already yield most of the achievable skill gain, because externalization can only reorganize patterns some trajectory already exhibits.

• An evolved environment produces genuinely harder but still answerable questions. Evaluating the same Branchus checkpoint on questions generated by the original versus the evolved task-synthesis skill, selection accuracy drops from 0.60 to 0.53 and Brier worsens from 0.52 to 0.58, while AUROC stays essentially flat (0.678 to 0.68) — difficulty rose without the questions degenerating into coin flips.

Methodology in Plain English

The authors treat forecasting as a gymnasium rather than a test. A module called Prometheus continuously reads the news and writes questions about what will happen next, with answers that only exist tomorrow. An agent called Branchus attempts those questions, and every attempt leaves behind a full record of what it searched for and how it reasoned.

The key engineering trick is splitting the agent in two. One half does the searching and is allowed to touch the live web, but only while the question is still open. The other half does the predicting and never sees the web — only the cached evidence and reasoning the first half produced. Because that cache is frozen before answers become public, the team can replay it many times for training without the model accidentally learning to look up the answer. The web-facing half never gets trained, which also blocks a shortcut where a model could learn to generate evidence that conveniently entails the answer.

With clean replays in hand, the authors push the same experience in three directions. They fine-tune the predicting half against resolved outcomes with reinforcement learning and with rejection fine-tuning on successful traces. Separately, they have a model read the trajectories and write down reusable procedures — explicit, inspectable forecasting skills — which are pasted into the prompt at inference time, costing no training at all. Third, and most unusually, they read the trajectories for what failed, distil each recurring failure into a "task-synthesis skill," and hand that to the question generator so the next batch of questions specifically targets the agent's weak spots.

Evaluation is kept live throughout. FutureX is an external weekly benchmark spanning four difficulty levels (L1 to L4), with multiple-choice, ranking, and numerical questions across finance, politics, sports, natural disasters, and technology, scored with its official sliding-window Bradley–Terry model. The self-synthesized evaluation uses Prometheus with its adversarial evolution disabled so the test stays unbiased, asks Codex to retrieve realized outcomes from the open web, and averages over several consecutive days because a single day is too small and variable to rank systems reliably. Alongside accuracy, the authors report Brier score, AUROC, relative WIS, MAPE, and an explicit calibration check comparing predicted probabilities to empirical frequencies.

Why This Matters

Impact on research. The paper reframes self-improvement as something that requires an environment, not just a better training recipe. Most prior work pushes a single lever — weights, prompts, or memory. This work argues that sustainability comes from the environment supplying fresh, verifiable, progressively harder experience, and it demonstrates a measurable loop in which parameters, skills, and environment each evolve from one shared substrate. It also offers a methodological point: static benchmarks cannot measure a system's rate or ceiling of improvement, because they cannot renew themselves.

Real-world applications.

• Forecasting and decision support: finance, geopolitics, sports, and disaster-response teams that need calibrated probability estimates rather than one confident answer.
• Continuous-monitoring systems: any deployment facing distribution drift — fraud detection, demand planning, clinical risk — where the world changes and a frozen model silently decays.
• Agent platforms that must improve without labeled datasets: settings where outcomes arrive naturally and can serve as free training signal.
• Cheap capability bootstrapping: the finding that skill distillation delivers much of the benefit at inference-only cost suggests a route to improvement for teams without large training budgets.

Industry relevance. The paper's cost analysis is directly actionable: skill extraction costs roughly 303,135 tokens per trajectory (14.96 calls) with no gradient updates, and saturates after about 88 trajectories, while parameter updates cost more but reach behaviours the current policy never exhibits. The suggested division of labour — externalize skills every round, reserve parameter updates for occasional longer bursts, and spend the saved compute on making the environment harder — is a concrete engineering pattern for anyone running an agent in a live setting.

Future Directions

• Measuring the ceiling of recursion. The authors state that improvement remains bounded by the quality and quantity of available experience, and that stability under non-stationary drift is not guaranteed. How far the loop can run, and when it flattens, is left open.
• Cheaper long-horizon verification. The paper flags long-horizon evaluation as expensive; the one-day resolution horizon used here is a deliberate workaround, and extending the loop to slower-resolving events is unresolved.
• Transferring the loop beyond forecasting. The authors argue the same construction carries over to any setting that is grounded, verifiable, and non-stationary, which invites testing in other domains with self-resolving outcomes.
• Jointly improving base model and agent. The finding that model capability and agent design are complementary rather than substitutable suggests future gains require co-developing the two rather than scaling either alone.

Target Audience

Researchers and engineers working on LLM agents, self-improving systems, and continual or online learning; forecasting and prediction-market practitioners interested in calibrated probabilistic output; and teams deploying agents in live, drifting environments who need improvement signal without hand-labeled data. Readers new to reinforcement learning will find the high-level argument accessible, but the results sections assume comfort with agent architectures and evaluation metrics such as Brier score, AUROC, and relative WIS.

Authors’ abstract

Most contemporary artificial intelligence (AI) systems are released with frozen parameters and improve through external updates, whereas an ideal system would keep improving from its own experience, a capacity we call recursive self-improvement (RSI). Yet studying RSI requires an environment that is continuously changing, objectively verifiable, and naturally grounded, posing a fundamental challenge for agent evolution. Here we introduce DeepEvolve, a sustainable evolution agent system grounded in future event prediction, in which agents continuously interact with a dynamically evolving environment and improve by internalizing and externalizing the experience gained through these interactions. A living environment synthesizes verifiable tasks, and a recursive-evolution engine turns the reasoning trajectory behind each prediction back into capability: decoupling search from prediction adapts parameter-level evolution to the contamination-prone live environment, while distilling reusable skills lets the agent evolve from the limited live trajectories such an environment affords. On the external FutureX leaderboard, which evaluates predictions against real-world outcomes and is updated weekly, our self-improved forecaster ranked first across all systems over three consecutive weeks, achieving up to a 195% improvement in rating. On our synthesized evaluation, it ranked among the top three across four base LLMs, outperforming the strongest baseline by up to 10.7 Bradley--Terry points. More broadly, these results suggest that a system built for self-improvement is best understood as an agent that co-evolves with a genuinely living, open-ended environment, pointing to a shift in AI from training static models on fixed data toward cultivating agents that grow stronger from their own experience, with such an open world as the indispensable basis of that growth.