AlphaBet: Reinforcement Learning for LLM Event Forecasting Agents

AI summary

Overview

Research area: Machine learning / LLM agents — specifically agentic reinforcement learning for probabilistic event forecasting.

Technical level: Advanced. The paper combines tool-using LLM agents, reinforcement learning (GRPO), and proper scoring rule theory with formal propositions and proofs.

Scope: An outcome-based RL framework that trains a single LLM policy to search for evidence and report probabilities at multiple pre-resolution dates for the same unresolved event, evaluated on Forecast-Dojo and FutureX with GLM-4.5-Air.

What This Paper Is About

Forecasting an unresolved event — an election, a policy decision, an economic indicator — means finding relevant evidence and deciding how strongly that evidence supports each possible outcome. Existing training methods for LLM forecasters usually hand the model a preassembled pile of evidence at a single prediction date, so the act of finding evidence is never part of what gets trained. The paper's goal is to bring evidence acquisition inside the learning loop: train one policy that searches, reads, computes, and reports probabilities at several dates before an event resolves, with each dated forecast scored against the eventual outcome.

Key Contributions

1. AlphaBet, an agentic RL framework for longitudinal forecasting. A single tool-using LLM policy revisits each training event at multiple pre-resolution dates, actively searches and reads date-restricted evidence, and reports a probability distribution over outcomes. Each dated report is scored against the eventual outcome, so one resolved event supervises decisions under several different information states. Cross-date memory is optional (memory-free vs. memory-on variants with a belief notebook).

2. A regret decomposition for outcome-based forecasting. Proposition 1 shows that minimizing forecasting regret is equivalent to minimizing the sum of an information gap (between the Bayes-optimal forecast given all admissible dated information and the forecast given the evidence the agent actually acquired) and a reporting gap (between that acquired-context target and the probability the agent reports).

3. A proper-scoring analysis of GRPO normalization. Proposition 2 extends earlier binary-outcome analysis to multi-class proper scores: at a fixed acquired context, mean-centering preserves the expected score-only reporting gradient up to a positive factor, whereas within-group standard-deviation normalization reweights outcome classes and induces a policy-dependent tilted reporting target unless the per-class reward scales are all equal.

4. Empirical validation on two benchmarks. Multi-date training improves forecasting on both Forecast-Dojo and FutureX, controlled ablations attribute the gains to multi-date training and to both better evidence acquisition and better probability reporting, and GRPO ablations support the normalization analysis.

Main Findings

• Headline improvements on Forecast-Dojo: the strongest AlphaBet variant raises GLM-4.5-Air accuracy from 41.6% to 50.5% and lowers the Brier score from 0.731 to 0.626.

• Headline improvements on FutureX: accuracy rises from 51.3% to 57.9% and Brier falls from 0.612 to 0.528.

• Competitive positioning: the trained model outperforms forecasting harnesses and prior RL-based training methods, and remains competitive with substantially larger off-the-shelf models.

• Multi-date training beats single-date training consistently across the experiments.

• Both gaps shrink: controlled evidence–reader swaps show that gains come from improvements in both information acquisition and probability reporting, not from one alone.

• Standardization hurts probabilistic quality: GRPO ablations show that within-group standard-deviation normalization degrades proper-scoring-rule performance while leaving top-outcome accuracy largely unchanged; removing it preserves the intended training signal.

• Setup details: base model GLM-4.5-Air (106B-A12B), public checkpoint released 07-28-2025; training on Forecast-Dojo's split of 1,028 binary and multi-option events (3,965 event–date tasks) from Polymarket with forecast dates spanning 08-01-2025 to 02-28-2026. Because all training events postdate the checkpoint release, their outcomes cannot be directly memorized from pretraining data.

• Evaluation window and corpus: Figure 1 describes evaluation over March 2026 to May 2026; retrieval uses a temporally indexed CC-News corpus spanning 03-2025 through 02-2026, restricted at each date to articles available by then.

• Not reported in the provided content: the numeric values inside Table 1 (including confidence intervals and the Info-α column) are not shown in the truncated text, and per-variant breakdowns of the three variants are not given numerically here.

Methodology in Plain English

The researchers build a simulated forecasting environment. Each question has a fixed set of mutually exclusive outcomes and a resolution criterion. During the question's active period, the agent visits it at several scheduled dates, and at each date it may only see information published on or before that date — so the available evidence grows as the clock advances.

At each visit, the agent runs a loop: it thinks, then calls tools. Search returns the top-k relevant article snapshots for a query from the date-restricted corpus; Read pulls the full text of a previously found article; Compute runs model-generated Python for base rates, aggregation, or simulation. It ends the step by committing a probability distribution over outcomes. In the memory-on variant it also writes a belief notebook summarizing its current view and unresolved questions, carried to the next date; in the memory-free variant each date starts fresh.

Training uses the eventual outcome as the reward signal at every date. The reward is a proper scoring rule (negative Brier score in the main experiments) plus a small format reward that checks the answer and notebook structure and whether tool calls use recognized names. The format reward is at most zero and vanishes for valid outputs, so it does not change the forecasting optimum.

For optimization they use GRPO, which compares a group of rollouts and builds relative advantages — but they drop the division by the within-group standard deviation, keeping only mean-centering. Their argument: the normalization scale depends on both the current policy and the realized outcome, so it rescales different outcome classes differently and shifts the probability the model is effectively being trained to report. They prove this for the multi-class case and then test it empirically.

Groups are formed per question and date for memory-free training; for memory-on training, G complete episodes are sampled per question and grouped by date across episodes. Rewards are immediate per-date rewards with no cross-date credit assignment, which the authors flag as a limitation of the memory-on update.

Why This Matters

Impact on research. Most prior outcome-based forecasting work trains only the reporting step, from fixed evidence at one date. This paper argues formally and empirically that the objective also covers the search step, and that a single resolved event can supervise many dated decisions. The GRPO normalization result is a general caution for anyone using group-relative RL with proper scoring rules: a design choice commonly treated as a harmless variance-reduction trick can silently change the target distribution.

Real-world applications:

• Prediction markets and trading desks — pricing contracts on policy, election, and economic events that stay open for weeks or months.
• Policy and intelligence analysis — maintaining and updating probability estimates as classified or public information accumulates.
• Corporate and financial risk monitoring — tracking regulatory decisions, earnings-relevant events, or supply-chain disruptions over time.
• Journalistic and research forecasting — supporting longitudinal, evidence-updating forecasts rather than one-shot predictions.

Industry relevance. The setup is directly deployable in any setting where a model must gather its own evidence under time constraints and commit to calibrated probabilities; the paper's environment is built on a replayable, temporally indexed benchmark with real market questions, and the trained model is compared against both harnesses and larger off-the-shelf models.

Future Directions

• Cross-date credit assignment. The authors explicitly name this as future work: currently earlier notebook actions that shape later forecasts are not assigned those future rewards, so the memory-on update is a per-date surrogate rather than a full episode-return gradient.

• Calibration of the complete trained agent. The normalization result characterizes only the local, score-only reporting update at a fixed context, not the calibration of the full trained policy — leaving open how standardization affects the end-to-end agent.

• Beyond negative Brier. The main experiments use the negative Brier score; the paper notes that exact properness holds for the ideal log score under finite expectation but not for its probability-floored implementation on the full simplex, leaving room to study alternative scoring implementations.

• Memory design. The gap between memory-free and memory-on forecasting, and what a belief notebook should contain and how it should be trained, is only partially explored.

Target Audience

Researchers and practitioners working on LLM agents, reinforcement learning from outcome feedback, and probabilistic forecasting — particularly those building tool-using agents that must retrieve evidence under date or recency constraints. It is also relevant to anyone applying GRPO-style group-relative optimization to tasks scored by proper scoring rules, and to forecasters interested in longitudinal rather than single-shot prediction. Readers need comfort with probability, scoring rules, and policy-gradient RL to follow the propositions, though the framing problem and empirical results are accessible at an intermediate level.

Authors’ abstract

Forecasting an unresolved event requires finding relevant evidence and deciding how strongly it supports each outcome. Yet outcome-based training of Large Language Model (LLM) forecasters often begins with preassembled evidence at a single prediction date, leaving evidence acquisition outside the learning loop. We introduce AlphaBet, an agentic reinforcement learning (RL) framework that trains a single LLM policy to search, reason, and report probabilities across multiple pre-resolution dates. Each dated forecast is scored by a proper scoring rule against the eventual outcome, allowing one resolved event to supervise decisions under different information states. We decompose forecasting regret into an evidence-acquisition gap and a probability-reporting gap, and show that Group Relative Policy Optimization (GRPO)'s standard-deviation normalization can distort the reporting target under multi-class proper scores. On GLM-4.5-Air, AlphaBet reduces the Brier score from 0.731 to 0.626 on Forecast-Dojo and from 0.612 to 0.528 on FutureX. Controlled ablations attribute these gains to multi-date training and show that they come from improvements in both information acquisition and probability reporting. Our empirical results validate that GRPO's std normalization degrades proper-scoring-rule performance while leaving top-outcome accuracy largely unchanged.