Axion 1.0: Open Agent Foundation Models, Simplified

AI summary

Overview

Research area: Artificial Intelligence — specifically agent foundation models: large language models trained to reason, call tools, search the web, and act over long multi-step trajectories rather than just produce single responses. The work sits at the intersection of post-training (supervised fine-tuning and reinforcement learning), synthetic data construction, and large-scale MoE training infrastructure.

Technical level: Advanced. The paper assumes familiarity with SFT, GRPO-style reinforcement learning, mixture-of-experts (MoE) parallelism, and agentic evaluation harnesses, although its central argument is stated in plain terms.

Scope (one sentence): The paper introduces Axion 1.0, a family of openly released agent models (35B-A3B and 122B-A10B, plus 4B and 9B SFT-only variants) trained with a deliberately minimal three-stage recipe — two SFT stages plus RL — and argues that careful data construction, not a longer training stack, is what produces strong agentic capability.

What This Paper Is About

Modern agent models are typically built with increasingly elaborate pipelines: continued or mid-training, multiple supervised stages, capability-specific data mixtures, preference optimization (DPO), reinforcement learning, and on-policy distillation (OPD). The authors ask how much of that complexity is actually necessary. Their goal is to build an agent foundation model that is both simple (two lightweight SFT stages followed by RL, with no mid-training, DPO, or OPD) and fully open (releasing data construction, generation, filtering, verification, prompts, and training code, not just weights).

Key Contributions

1. A three-stage recipe for agent foundation models. Axion 1.0 is trained with two lightweight SFT stages followed by reinforcement learning, explicitly omitting dedicated mid-training, direct preference optimization (DPO), and on-policy distillation (OPD).

2. A model-aware pipeline for self-contained reasoning data. Seed problems from public course materials, competitions, and reasoning datasets across mathematics, physics, chemistry, biology, and computer science are rewritten into harder problems, then strengthened using the base model's own successful solutions through one of four transformations (Composition, Parameterization, Range Shift, Case Analysis).

3. A scalable pipeline for automatically verifiable tool-dependent tasks. Questions are built by a filter–select–compute–verbalize procedure over structured public data sources (biology, chemistry, mathematics, geography, aviation), so answers are computed deterministically before any model attempts them, and construction-time databases are withheld during solving to force genuine information seeking.

4. Openness beyond weights. The authors release the complete development recipe — data construction, generation, filtering, verification, prompts, and training code — alongside the models (project page https://axion1-0.github.io/, code https://github.com/bytedance/axion1_0).

Main Findings

• Competitive agentic performance from a compact recipe. On full Humanity's Last Exam (HLE), Axion 1.0-35B and Axion 1.0-122B score 52.2% and 59.0% with restricted website access (for example, Hugging Face blocked); without those restrictions they score 59.1% and 64.8%.

• Gains over a similarly sized open agent model. Compared with Agents-A1, Axion 1.0-35B improves by 4.6% on HLE (52.2% vs. 47.6%), 5.8% on SEAL-0 (62.2% vs. 56.4%), and 7.0% on FrontierScience Olympiad (86.0% vs. 79.0%).

• Strong showing on research-level scientific reasoning. On FrontierScience Research, the 35B and 122B models score 56.7% and 66.7%, versus 40.0% for Agents-A1 and 26.7% for GPT-5.5; the 122B model exceeds GPT-5.6 Sol's 63.3% by 3.4%. Corresponding average rubric scores are 6.67 and 7.09.

• A weaker spot on human-difficult biology. On BioMysteryBench (human-difficult subset), Axion 1.0-122B reaches 41.7%, compared with 47.1% for GPT-5.6 Sol and 46.1% for Claude Fable 5.

• Bigger models help on five of six benchmarks. Moving from 35B to 122B improves five benchmarks, with the largest gains on FrontierScience Research (+10.0%), HLE (+6.8%), and BioMysteryBench (+6.4%), and a smaller gain on HiPhO (+0.5%). SEAL-0 is the exception: 62.2% for the 35B model and 58.6% for the 122B model.

• Most capability comes from the two SFT stages. For the 35B model, Stage I lifts HLE-130 from 47.7% to 59.2%, HiPhO from 38.6% to 45.9%, and FrontierScience Olympiad from 71.0% to 83.0%; Stage II then lifts BioMysteryBench from 17.6% to 35.3% and FrontierScience Research from 48.3% to 58.3%.

• RL adds targeted, not universal, gains. RL raises the 35B model from 66.2% to 70.0% on HLE-130 and from 53.2% to 62.2% on SEAL-0, but FrontierScience Research declines from 58.3% to 56.7% and HiPhO dips from 46.8% to 45.7%. For the 122B model, RL lifts BioMysteryBench from 35.3% to 41.7%, HLE-130 from 68.5% to 74.6%, and SEAL-0 from 53.2% to 58.6%.

• One regression during training is reported. The 35B model's BioMysteryBench score decreases from 23.5% to 17.6% after SFT Stage I, before recovering in later stages.

• The recipe transfers to weaker initializations and smaller models. Applying the same two SFT stages to Qwen3.5-35B-A3B and Qwen3.6-35B-A3B shrinks the HLE-130 gap from 9.2% to 0.8%, and both reach 35.3% on BioMysteryBench (though Qwen3.5 ends higher on SEAL-0, 57.7% vs. 53.2%). Among the SFT-only smaller models, the 9B model beats the 4B model on all six benchmarks, including HLE-130 (63.1% vs. 53.1%) and BioMysteryBench (23.5% vs. 11.8%).

• Infrastructure engineering yields measurable stability gains. Rollout routing replay (R3) on Qwen3.5-122B-A10B reduces mean sequence-average log-probability gap from 0.044 to 0.042, maximum gap from 1.799 to 1.282, mean gradient norm from 1.998 to 1.922, mean PPO KL from 0.0413 to 0.0396, and clipping fraction from 3.86% to 3.68%, accompanying higher HLE-130 (74.6% vs. 72.3%) but lower SEAL-0 (58.6% vs. 63.1%).

Methodology in Plain English

Step 1 — Build hard reasoning problems instead of collecting them. The team starts from existing question–answer pairs, but never trains on them directly. A generator model rewrites each seed into a new problem that depends on a quantity from the original solution (called composition). The base model then attempts each rewritten problem with Python available but search and webpage visits disabled. If it fails, the problem goes straight into the pool. If it succeeds, the generator is shown that successful solution and applies one of four transformations to make the problem harder: add another dependent step, replace a fixed number with a symbolic parameter, shift the parameter range so shortcuts stop working, or introduce cases with different correct answers. A reference answer is accepted only when GPT-5.5 solves the problem three independent times and all three agree; answers are recomputed after every rewrite rather than inherited. A separate model then solves the accepted problems and only correct, complete trajectories are kept.

Step 2 — Build tasks that can only be solved by looking things up. For tool use, the authors start from structured public databases. A program applies filters, selects exactly one entity by an extreme-value criterion (requiring a margin over the runner-up to avoid ambiguity), retrieves further attributes, and computes a final answer with a deterministic formula. A language model then phrases this as a natural-language question while preserving every constraint, value, and unit. During solving, the construction databases are blocked — general search, webpage visits, and Python remain available — so the model must genuinely retrieve and combine evidence. Questions answerable closed-book are probed and discarded. Intermediate entities and values are kept as verification targets, never shown to the solver.

Step 3 — Train in three stages. Stage I fine-tunes on the self-contained reasoning data; Stage II continues from that checkpoint on the tool-dependent data; RL then starts from the Stage II checkpoint. Supervision is assistant-only: the full interaction (including tool observations) is serialized, but loss is computed only on the model's own reasoning, tool calls, and answers.

Step 4 — Reward progress, not just answers. RL uses GRPO in the verl framework with a reward of 1 for a correct final answer; otherwise the reward is max(0, 0.3P − 0.1(1−A)), where P is verified intermediate progress and A marks whether a valid final answer was submitted. Progress is credited in dependency order: a later step counts only if all its prerequisites are verified, so if only the first and third of three checks pass, P = 1/3. Tasks with no intermediate targets get P = 0. Questions are kept for RL only if, out of four sampled trajectories, at least one is correct and the four rewards have nonzero variance.

Step 5 — Engineer the plumbing. SFT uses Megatron-FSDP inside ms-swift's Megatron backend, with expert parallelism degree 8 and tensor-parallel degree 1, plus context parallelism (Ring Attention for full-attention layers, a Ulysses-style all-to-all for gated-delta-net layers) at context-parallel degree 4 and micro-batch size 2 on 128K-token sequences. RL runs asynchronously: vLLM rollout engines on separate GPUs from the Megatron-Core trainer, weight transfers via NCCL every two optimizer steps (45–60 seconds for the 122B model), and trajectories more than 11 parameter versions old discarded. Two consistency tricks matter: Token-In-Token-Out avoids re-tokenization drift, and R3 replays the MoE expert choices made during rollout inside the trainer's forward pass.

Why This Matters

Impact on research. The paper attacks a real reproducibility problem: many "open" agent models release weights while hiding data construction, filtering, verification, prompts, and training procedures — precisely the parts that determine agentic capability. By releasing the whole recipe and showing that two SFT stages plus RL suffice, the authors provide a baseline that others can actually reproduce, inspect, and extend, and they challenge the assumption that more post-training stages necessarily buy more capability. Their emphasis on automatically verifiable task construction also offers a template for generating agentic training data at scale without human annotation.

Real-world applications (implied by the capabilities the paper trains and evaluates):

• Information-seeking assistants that must locate facts across multiple databases, cross-reference entities, and compute derived quantities with correct units.
• Scientific and technical analysis tools, given the evaluation on Humanity's Last Exam, HiPhO, FrontierScience, and real biological data in BioMysteryBench.
• Research agents that combine web search, scholarly retrieval, webpage visits, and Python execution over long, multi-step trajectories.
• Reproducible in-house agent training for organizations that cannot afford closed frontier-model pipelines, since the full data and training recipe is public.

Industry relevance. The reported training cost is modest for models of this size: all SFT runs use a 128K context window on 32 GPUs, with the 122B model's two stages totaling 7 h 30 min (240 GPU-hours, 113 GiB peak memory per GPU) and the 35B model's totaling 2 h 48 min (approximately 90 GPU-hours, 101 GiB peak memory per GPU). That makes the recipe plausible for teams outside the largest labs. The systems contributions — FSDP over pipeline parallelism, expert parallelism, context parallelism for 128K-token agent trajectories, asynchronous rollout/trainer separation, TITO, and R3 — are directly reusable by anyone training MoE agents. The authors are affiliated with ByteDance, Duke University, University of Maryland, College Park, and University at Buffalo, and the project is released under CC BY-NC-ND 4.0.

Future Directions

• When does RL actually help? RL consistently improved HLE-130, SEAL-0, and FrontierScience Olympiad across the two main models, but its effects on BioMysteryBench, HiPhO, and FrontierScience Research varied by model size — including small regressions for the 35B model. Isolating which task properties make RL pay off is an open question.

• Closing the BioMysteryBench gap. The 122B model trails GPT-5.6 Sol (41.7% vs. 47.1%) and Claude Fable 5 (46.1%) on human-difficult biological data analysis, and the 35B model regresses on this

Authors’ abstract

We introduce Axion 1.0, a fully open agent foundation model that combines strong and broad agentic capabilities with a remarkably simple training recipe. Axion 1.0 is trained in only three stages: two lightweight supervised fine-tuning stages followed by reinforcement learning, without dedicated mid-training, direct preference optimization (DPO), or on-policy distillation (OPD). The first SFT stage develops strong self-contained reasoning through model-aware data construction, where we synthesize challenging problems and adversarially strengthen them based on the base model's own successful solution strategies. The second stage extends these capabilities to tool-dependent reasoning through a scalable pipeline for constructing diverse and automatically verifiable tasks grounded in real-world data sources and deterministic computation. Finally, reinforcement learning further strengthens long-horizon reasoning and interaction with external tools and environments. Despite this compact training recipe, Axion 1.0 achieves strong performance across a broad suite of agentic benchmarks and is competitive with leading commercial models on several of them. Equally importantly, Axion 1.0 is open beyond model weights: we release the complete recipe, including data construction, generation, filtering, verification, prompts, and training code, enabling the entire development process to be reproduced and extended. Axion 1.0 demonstrates that strong agent foundation models can be both open and simple, offering a practical and reproducible path toward advancing open agentic intelligence.