Marathoner: Ultra-Long-Horizon Autonomous Intelligence

AI summary

Overview

• Research area: Autonomous coding agents and agentic post-training (computer vision category on arXiv, but the content is software-engineering agent training and evaluation).
• Technical level: Advanced.
• Scope: The paper introduces Marathoner, a 9B open-source agentic model trained with a post-training pipeline (task synthesis, rejection-sampling finetuning, reinforcement learning) to sustain ultra-long-horizon execution, evaluated on 5 software-engineering benchmarks.

What This Paper Is About

Strong proprietary models such as Fable 5 and GPT-6-Astra can reportedly work continuously for days on hard problems, but the training methods behind that persistence are not disclosed, and open-source models lack the capability entirely. The paper's goal is to build that same persistent, "ultra-long-horizon" execution ability into an open base model through a single post-training pipeline, and to test whether a smaller open model can then compete with proprietary systems on very long tasks.

Key Contributions

1. Marathoner-9B, an ultra-long-horizon agentic model. The authors report the model can continuously execute for 10+ hours and conduct 1000+ tool calls on highly challenging tasks.
2. A comprehensive post-training pipeline consisting of Ultra-Long-Horizon Task Synthesis, rejection sampling finetuning, and reinforcement learning, built on 10,000 GitHub repositories and 100,000 synthesized tasks.
3. Multi-Task Chaining, which chains multiple synthesized tasks (5 random tasks) into a single Frontier-difficulty task requiring coordination across multiple large repositories and continued progress after intermediate objectives are met.
4. Later Stage Bonus Reward, a reinforcement-learning reward that grants an additional 0.5 bonus when an LLM judge flags exceptionally valuable agent operations during the later half of an execution trajectory, plus superior results across 5 ultra-long-horizon benchmarks.

Main Findings

• Large gains over the base model: Marathoner-9B (evaluated with Claude Code) scores 26.4 on FrontierSWE, 34.7 on NL2Repo, 8.2 on SWE-Marathon, 57.2 on Terminal Bench 2.0, and 77.5 on SWE-Bench Verified, versus base model Qwen3.5-9B at 10.2, 17.9, 0, 27.3, and 43.8.
• Surpasses a proprietary model on two benchmarks: Marathoner-9B exceeds Gemini-3.1-Pro on FrontierSWE (26.4 vs 25.9) and on SWE-Marathon (8.2 vs 5.9).
• Still behind the strongest proprietary systems: GPT-6-Astra (Codex) reaches 89.1 / 78.2 / 48.7 / 87.4 / 90.7 and Claude Fable 5.1 (Claude Code) reaches 86.2 / 73.2 / 42.8 / 85.9 / 91.8 across the same five benchmarks, well above Marathoner-9B.
• Real execution statistics: On FrontierSWE the model averages 3.56 hours, 426.2 steps, and 648.3 tool calls per run; on Terminal Bench 2.0 it averages 0.58 hours, 104.8 steps, and 239.4 tool calls.
• Environment scaling helps: Increasing RL training tasks from 1,000 to 8,000 produced consistent, substantial improvement on FrontierSWE and Terminal Bench 2.0, which the authors call a critical path for raising the ceiling of agentic performance.
• Stable RL optimization: RL reward rises consistently across training while policy entropy remains stable, indicating no excessive exploration or policy collapse.
• Ablations support each design: Starting from Marathoner (vanilla) at 22.7 on FrontierSWE and 51.3 on Terminal Bench 2.0, Multi-Task Chaining gives 24.9 / 54.8, Diverse-Harness Trajectory Generation gives 24.1 / 53.7, and Later Stage Bonus Reward gives 24.6 / 54.4.
• Task mix skews hard: Of 100,000 mined major release PRs, the Easy/Medium/Hard ratio is 2:3:5 — 20,000 Easy, 30,000 Medium, and 50,000 Hard tasks.

Methodology in Plain English

The authors build training data out of real software changes. They collect 10,000 diverse GitHub repositories (frontend, backend, CUDA and database kernels, mature codebases such as SciPy, trending repos, and LLM repos such as vLLM and SGLang) and mine 100,000 "major release" pull requests that add substantive new functionality. Tasks are sorted by how much new code the PR adds: Easy (100–200 lines), Medium (200–1000), and Hard (1000+ lines). Each task bundles a codebase checked out at the commit just before the PR, an instruction (taken from the PR's first comment, the release notes, or generated by an LLM from the PR diff), a reward verifier made of unit tests, and a Dockerfile. Fail-to-pass tests come from tests introduced in the PR; pass-to-pass tests are the pre-existing test suite. Rewards are binary: 1 only if all unit tests pass. The sandbox is a clean Ubuntu 24.04 image with 4 CPUs, 10 GB memory, and 40 GB storage, with no pre-installed packages, so the agent must configure environments itself.

To push difficulty higher, Multi-Task Chaining randomly combines 5 atomic tasks into one task whose software spans multiple repositories. For rejection sampling finetuning, the teacher model Kimi K3 generates trajectories with several different agent harnesses — Claude Code, Codex, and OpenClaw — to prevent overfitting to one interface. Only trajectories scoring reward 1 are kept (14,159 from Claude Code, 13,583 from Codex, 13,078 from OpenClaw; 40,820 total), and the base model is finetuned on them at a 256k sequence length.

In the reinforcement learning stage, the cold-started policy performs real rollouts in independent sandboxes with a harness randomly chosen per task, using GRPO as the algorithm. A separate LLM (Qwen-3.8-Max) summarizes each finished trajectory into phases and judges whether any phase in the latter half contains exceptionally valuable work, such as fixing hidden bugs or making key optimizations; if so, the trajectory earns 0.5 extra reward on top of the binary correctness reward. Data wise, 50,000 tasks feed the RFT stage, 15,000 tasks are chained into 3,000 Frontier tasks, and RL uses 5,000 synthesized tasks plus the 3,000 chained ones for 8,000 total. Training uses 32 H800 GPUs.

Why This Matters

Impact on research. The paper argues that ultra-long-horizon execution is dominated by undisclosed proprietary methods, and shows a recipe that open-source models can follow. It also provides a synthetic-task-generation scheme (including test-based verifiers) and a process-level reward that rewards late-stage progress rather than only final correctness.

Real-world applications:

• Large-scale codebase migrations and refactors that span many repositories and take days of coordinated work.
• Long-horizon debugging and performance optimization, such as iteratively tuning low-level kernels.
• Terminal-centric system administration and environment setup, the target of Terminal Bench 2.0.
• Building entire code repositories from natural-language descriptions, the target of NL2Repo.

Industry relevance. The results suggest a small open model (9B) post-trained on realistic agentic tasks can beat some strong proprietary models on two benchmarks for a fraction of the scale, which matters for teams that want autonomous coding agents they can train and deploy themselves. The scaling result — that going from 1,000 to 8,000 RL environments keeps improving performance — is an argument that investing in environment infrastructure, not just model size, is where agent capability comes from.

Future Directions

• The authors do not report benchmark-specific optimization, so an open question is how far the same unified pipeline scales with more environments, larger base models, or longer execution budgets.
• Marathoner-9B still trails GPT-6-Astra and Claude Fable 5.1 by large margins on FrontierSWE, NL2Repo, SWE-Marathon, and Terminal Bench 2.0; narrowing that gap is an obvious next step.
• The paper does not report how the difficulty mix (Easy/Medium/Hard, and chained Frontier tasks) should be tuned, nor how varying the number of chained tasks beyond 5 affects capability.
• The Later Stage Bonus Reward depends on an LLM judge, so the reliability and cost of that judge, and whether process-level rewards can be obtained without a proprietary model, remain open.

Target Audience

Researchers and engineers working on autonomous coding agents, agent post-training (SFT and RL), and agentic benchmark design. It is most useful to readers already familiar with reinforcement learning from verifiable rewards, tool-using agents, and software-engineering benchmarks, since the paper assumes background in GRPO, rejection sampling, and harness-based evaluation.

Authors’ abstract

Humans naturally possess the ability to work persistently toward long-term goals. Given a challenging task, humans can continuously work for months or even years to accomplish a specific objective. In this paper, we propose Marathoner, an autonomous agentic model possessing the ability of ultra-long-horizon execution. Specifically, we propose a comprehensive post-training pipeline to instill this critical capability into base model. For Ultra-Long-Horizon Task Synthesis, we leverage major release PRs containing 1000+ lines of new code from diverse GitHub repositories as the primary source for synthesizing challenging task-level data. Additionally, we introduce Multi-Task Chaining, which chains multiple generated tasks into a single more challenging task, enabling the synthesis of tasks with frontier-level difficulty. For rejection sampling finetuning, we combine strong teacher model with diverse harnesses to generate trajectories on our synthesized tasks and conduct supervised finetuning on base model with rejection sampled trajectories. For reinforcement learning, cold-started model performs real-world execution through harnesses in independent sandboxes during rollout process, effectively facilitating the acquisition of genuine ultra-long-horizon execution capability. We further propose a novel reward strategy, Later Stage Bonus Reward, which explicitly encourages model to perform meaningful maneuvers during later stages of execution. Through extensive evaluation on 5 benchmarks containing ultra-long-horizon tasks, Marathoner achieves consistent and substantial performance improvements over base model and even surpasses performance of strong proprietary model. Further analysis shows that Marathoner can consistently work for 10+ hours and conduct 1000+ tool calls on highly challenging tasks.