Solvent: Long-Horizon Reasoning for Multi-Step Retrosynthesis

AI summary

Overview

• Research area: AI for scientific discovery, specifically LLM-based retrosynthetic planning (computer-aided synthesis planning) and long-horizon agentic reasoning with reinforcement learning.
• Technical level: Advanced. The paper assumes familiarity with retrosynthesis search (Retro*, MCTS, AND–OR trees), rejection-sampling fine-tuning, and group-relative policy optimization; the prose itself is readable, but the subject matter is specialist.
• Scope: An 8B-parameter LLM named Solvent that treats multi-step retrosynthesis as a multi-turn agent task — planning, checking precursors against a building-block inventory, and revising — trained through a two-round post-training curriculum, and evaluated on four retrosynthesis benchmarks against LLM and search-based baselines.

What This Paper Is About

Multi-step retrosynthesis asks a system to work backward from a target molecule through successive reactions until every starting material is a purchasable building block. Most AI systems solve this by pairing a single-step reaction predictor with an external search algorithm, which ties route recovery to that predictor's coverage and leaves the overall chemical strategy implicit. Solvent instead lets one LLM own the whole loop — propose a route, query an inventory of commercially available compounds, read the availability feedback, and revise — and shows that an 8B model trained this way can rival much larger frontier LLMs and outperform prior search-based planners.

Key Contributions

1. Reframing retrosynthesis as a long-horizon task. The paper recasts multi-step planning as iterative planning, external verification, and adaptive revision across turns, with all retrosynthetic decisions kept inside the model rather than delegated to an external search procedure or single-step model.
2. A two-round post-training curriculum for long-horizon reasoning. Round 1 builds a seed policy (teacher-distilled retrosynthetic knowledge, teacher-guided cold start on multi-turn trajectories, then GSPO reinforcement learning); Round 2 bootstraps a stronger starting policy via rejection-sampling fine-tuning, then applies a final RL stage with a success-gated reward.
3. Strong performance at 8B parameters. Solvent averages 61.86% success across four benchmarks, exceeding the similarly sized retrosynthesis-specialized LLM R³ by 42.83 percentage points and rivaling frontier trillion-parameter models.
4. Consistent gains over search-based planners across budgets. Against prior search-based systems, Solvent reaches an average success rate of 93.22% at a search budget of 100 and records the highest success rate across the evaluated budget range (N = 50 to 500).

Main Findings

• Headline accuracy: Solvent (8B) scores 61.86% average success over USPTO-190, ChEMBL-1000, Pistachio Reachable, and Pistachio Hard — 62.00, 78.00, 48.95, and 58.50 respectively. The abstract attributes a 2.1× performance boost to RL.
• Large margin over same-scale models: The 8B retrosynthesis specialist R³ averages 19.03% (19.00 Pistachio Hard, 26.67 Pistachio Reachable, 13.16 USPTO-190, 17.30 ChEMBL-1000). Other mid-scale baselines are far lower: Qwen3-8B 0.16%, Qwen3.5-9B 0.17%, and Qwen3.6-Flash (35B) 11.13%.
• Competitive with, not superior to, frontier LLMs: On the shared 400-target subset (marked † in the paper), GPT-5.5 averages 63.57%, Kimi K3 (2.8T) 63.03%, Grok 4.5 59.15%, Gemini 3.5 Flash 58.30%, and Claude Opus 4.8 56.54%, versus Solvent's 61.86% on the full test sets. Open-weight frontier models trail further (DeepSeek-V4-Pro 1.6T at 42.70%, Kimi K2.5 1T at 27.08%, GLM-5.2 753B at 3.73%).
• Best across search budgets: On USPTO-190 at N = 100, Solvent reaches 87.36% versus PDVN 72.11% and RETRO-R1 77.21%; at N = 500 it reaches 97.37% versus PDVN 92.11%. On ChEMBL-1000 at N = 100 it reaches 93.20% versus RETRO-R1 80.21%, and 98.20% at N = 500.
• Near-ceiling on Pistachio: Pistachio Reachable rises from 99.33% (N = 100) to 100.00% (N = 300 and 500); Pistachio Hard from 93.00% to 98.00%. The strongest baselines there are AOT* (98.70/99.30/99.30 and 86.00/89.00/93.00) and RootAligned with MCTS (99.30 across budgets and 83.00/85.00/85.00).
• Every training stage helps, and final RL helps most: Average success moves from 0.18% (initial backbone) to 6.26% (teacher-guided cold start), 22.82% (first-round RL), 29.19% (reasoning bootstrapping), and 61.86% (final-stage RL) — the last stage contributing the largest single jump.
• Interaction turns pay off: As the interaction horizon expands, Solvent's average success rate rises from 18.75% to 61.86%, and the measured benefit per additional turn increases across post-training stages.
• What final-stage RL learns first: Validation success climbs steadily, with chemical validity improving before stock completion — suggesting the model first fixes individual disconnections and only later assembles them into fully purchasable routes. Mean response length stays stable during training.
• Qualitative chemical reasoning: Case studies report Solvent devising protecting-group schemes and convergent routes where appropriate, and revising routes in response to inventory feedback.

Methodology in Plain English

The environment is deliberately simple: the model gets a target molecule and can either submit a proposed route or query an inventory of commercially available building blocks (eMolecules) to see whether a route's terminal precursors are purchasable. Success requires a valid route tree rooted at the target, all leaves in stock, and every reaction passing a chemical-validity check that combines plausibility with atom-source accounting.

Training happens in two rounds.

Round 1 — build a stable seed policy. First, a teacher model distills retrosynthetic knowledge from reference routes into structured analyses covering target-level strategy and each reaction step; analyses that don't faithfully preserve the original route are discarded. The authors fine-tune a retrosynthesis-pretrained 8B backbone on this corpus to get a "warmed-up" policy. Second, the teacher demonstrates multi-turn behavior — reasoning, action, observation — and only trajectories yielding valid solutions, consistent with the recorded inventory interactions, and chemically valid are kept. Third, reinforcement learning with GSPO stabilizes the behavior, since the teacher–student capability gap otherwise produces truncated responses and malformed inventory queries. The reward is 0.6 for solution validity, 0.2 for inventory verification, and 0.2 for format, with targets drawn from USPTO-full and any benchmark overlap removed.

Round 2 — bootstrap, then boost. The seed policy samples many trajectories per target, which are filtered for stock-completeness, consistency, and chemical validity, then scored LIMO-style along four axes (reasoning elaboration, self-verification, exploration, adaptive granularity) combined with a reasoning-style diversity term and an n-gram repetition penalty; only the highest-scoring trajectory per canonical target is kept. Fine-tuning the warmed-up policy on this curated set gives the bootstrapped policy — using the first-epoch checkpoint because later training sharply reduces entropy and harms exploration. A final GSPO stage then rewards route success at 0.8 and gates the verification and format terms on success, so credit flows to route construction, inventory interaction, and revision only when they jointly produce a successful route.

Evaluation uses a 32k-token output limit and up to five interaction turns for all LLMs; comparisons with search-based planners use aligned search budgets, where N counts explicit reaction-level construction before a successful route is found.

Why This Matters

Impact on research. The paper argues that retrosynthesis is a natural fit for long-horizon LLM agents — recursive subproblems, external checking, feedback-driven repair — yet had been underexplored in that framing. It shows an 8B model can absorb the planning role previously split between a single-step predictor, a value function, and a search algorithm, and it reports that the resulting proposals are more interpretable (protecting groups, convergent designs) than routes implicit in a search procedure. It also adds a data point to the debate about how much long-horizon agentic RL buys at small scale.

Real-world applications:

• Drug discovery and medicinal chemistry, where rapid route scouting for novel bioactive compounds shortens design–make–test cycles (ChEMBL-1000 is drawn from the ChEMBL bioactivity database).
• Process chemistry and route selection for scale-up, where alternative disconnections and protecting-group strategies need to be compared before committing to a synthesis.
• Automated or semi-automated synthesis planning platforms that need a planner able to negotiate with a live inventory of available building blocks rather than a fixed stock list.
• Training and benchmarking infrastructure for AI-for-chemistry: four standard benchmarks plus a released model and code lower the barrier for follow-up work.

Industry relevance. The cost profile matters: a competitive result from an 8B open-weight model, released with code and weights, is attractive to pharmaceutical and chemical companies that cannot send proprietary targets to closed frontier APIs. The explicit inventory-checking loop also maps onto real procurement constraints, which is where most planning systems fail in practice.

Future Directions

• Verification beyond in silico proxies. The authors state plainly that reliable route verification is open; current checks cannot establish experimental success, yield, selectivity, reproducibility, or operational safety.
• Closing the loop with wet labs. The proposed next step is connecting long-horizon agents to automated synthesis platforms so proposed routes and conditions are judged against measured outcomes and practical constraints, with experimental results guiding later planning cycles.
• Scaling the paradigm. The conclusion frames the approach as "a path to further scaling," leaving open how the curriculum behaves at larger parameter counts or with more capable backbones.
• Reward and curriculum design. Since the largest gains came from the final RL stage and from the bootstrapped initialization, questions remain about how to tune the reward gating, the target-difficulty curriculum, and the exploration–exploitation checkpoint choice more systematically.

Target Audience

Machine learning researchers working on agentic RL, long-horizon reasoning, and tool/environment interaction; computational chemists and cheminformaticians interested in LLM-based synthesis planning; and AI-for-science practitioners in industry who need strong planning performance from small, self-hostable models. Readers without chemistry background can follow the framing and results but will need the appendix and cited search-based literature to interpret the baseline comparisons.

Authors’ abstract

Multi-step retrosynthesis requires recursively transforming a target molecule through successive reaction steps into purchasable building blocks, and recent work has begun to draw on large language model (LLM) reasoning to tackle the task. In its latest evolution, LLM reasoning has entered the long-horizon regime: models pursue objectives across interdependent steps, interact with environments, and adapt to feedback. This shift has markedly advanced LLM capabilities, as illustrated by breakthroughs in AI-driven mathematical research. At this frontier, we introduce Solvent, an 8B LLM that successfully extends long-horizon reasoning to multi-step retrosynthesis. Through a two-round post-training curriculum, Solvent learns to interleave iterative synthesis planning, external availability verification, and adaptive route revision. Experiments show that, following a $2.1\times$ performance boost from RL, Solvent scores 61.86% across four benchmarks, surpassing mid-scale language models and prior search-based retrosynthesis systems, and rivaling frontier trillion-parameter LLMs.