Atlas: Orchestrating Heterogeneous Models and Tools for Multi-Domain Complex Reasoning

AI summary

Overview

• Research area: Natural language processing, specifically tool-augmented LLM agents, query-based model routing, and reinforcement learning for agentic reasoning.
• Technical level: Advanced. The paper assumes familiarity with LLM inference pipelines, embedding-space clustering, and policy-gradient RL (PPO).
• Scope: The paper introduces Atlas, a dual-path framework that jointly selects which LLM and which external tool should handle a given query, combining a training-free cluster-based router with an RL-trained multi-step router.

What This Paper Is About

As both the number of available LLMs and the number of external tools grow, choosing the right model-tool pairing for a query becomes a high-dimensional search problem. Existing routers pick models without tools, and existing tool-use frameworks call tools with fixed logic, so neither exploits the performance variations across heterogeneous model-tool pairs. Atlas addresses this by combining empirical, cluster-based routing for familiar domains with RL-driven multi-step routing for unfamiliar ones.

Key Contributions

1. A generalizable agentic framework (Atlas) that explicitly optimizes heterogeneous synergies between diverse LLMs and tools, enabling dynamic and adaptive tool invocation for complex reasoning tasks.
2. A dual-path design: training-free cluster-based routing for efficient selection using domain expertise, plus RL-driven multi-step routing that generalizes to unfamiliar tasks through iterative exploration.
3. Broad empirical validation: experiments across 9 tasks and 15 benchmarks showing Atlas outperforms closed-source models such as GPT-4o and existing routing methods on multi-domain tasks, with robust adaptability in multi-modal scenarios.
4. Evidence of extensibility: the RL path integrates newly added models and tools at inference time without retraining, improving average accuracy under an expanded routing pool.

Main Findings

• In-distribution gains: With training data available for all tasks, Atlas (cluster) reaches 63.5% average accuracy, surpassing the strongest baseline RouterDC (53.4%) by 10.1%. It exceeds GPT-4o (53.1%) and approaches GPT-4.1 (63.0%).
• Strong math results: Atlas (cluster) scores 40.0% on AIME25 and 82.5% on AMC, which the paper reports as +16.7% and +20.0% over RouterDC.
• Out-of-distribution generalization: Atlas (RL) maintains 59.4% average accuracy, 10.2% above Atlas (cluster) (49.2%) and 13.1% above RouterDC (46.3%). On AIME24 and AIME25, Atlas (RL) sustains 43.3% and 33.3%, versus 13.3% and 3.3% for the clustering method, described as a 10x difference.
• Closing the gap to proprietary models OOD: Atlas (RL) scores 43.3% on AIME24 and 42.0% on GPQA while using only 7B and 8B models, approaching or exceeding GPT-4o.
• Multi-modal orchestration: Across five visual datasets, Atlas achieves 68.9% average accuracy, outperforming the strongest single-tool baseline by 4.3%. It exceeds the best individual tool in each task category, for example surpassing Qwen3-Chart's 83.0% on ChartQA, and works around that tool's 50.2% on Geometry3K.
• Pool extension without retraining: Adding Llama-3.1-8B-UltraMedical, Qwen2.5-Math-7B-Instruct, and an Outcome Reward Model at inference time lifts Atlas (RL) from 59.4% to 61.7% (+2.3%), with AIME24 and AIME25 each gaining +6.7%. BertRouter gains only +1.5%, and RandomRouter degrades.
• Reasoning capacity boundary: RL training yields +23.0% absolute pass@1 improvement (36.4% to 59.4%). At pass@16, the upper bound is 63.1% (+3.7%), with per-task gains ranging from +6.1% to +35.0%.
• Adaptive compute allocation: API call counts rise for reasoning-intensive tasks like AIME25 and GPQA and stay minimal for retrieval tasks like WebQ and NQ, indicating task-adaptive budget use.
• Training stability: Adding the model selection reward produces faster convergence to a higher plateau, and entropy drops more sharply than the ablation group, indicating a shift from stochastic exploration to high-confidence decisions.
• Statistical significance: A Wilcoxon signed-rank test pairing Atlas against RouterDC across the Table 1 benchmarks yields p = 9.7 × 10⁻⁴ in-distribution (below the stated α = 0.05), with equivalent significance in the OOD setting. Across three repeated runs, Atlas (Cluster) scores 82.5 ± 2.5 on AMC and 91.5 ± 1.6 on HumanEval.

Methodology in Plain English

Atlas has two routes for picking a model-tool pair.

The first is a training-free cluster-based router. Every query is converted into a vector by a pre-trained encoder. Training queries are grouped into clusters so that queries with similar reasoning needs land together. For each cluster the system records, for every model-tool pair, how often it solved the query (empirical accuracy) and how much it cost (mean input and output token counts times their unit prices). These combine into a utility score, (1 − α) · Accuracy − α · Cost, where α trades off performance against expense. At inference, a new query is embedded, matched to its nearest cluster centroid, and the highest-utility pair in that cluster is executed. If no statistics exist for a cluster, a fallback strategy handles the query.

The second is an RL-driven multi-step router. Here the router is an agent operating over a maximum horizon. At each step the policy chooses between think (internal chain-of-thought) and route (invoking a specific model-tool pair and observing the result), accumulating context until it emits a final answer. The policy is trained with PPO using a composite rule-based reward: a format reward that checks tag integrity, invocation syntax, mandatory reasoning, a single answer block, and search/information consistency; a binary outcome reward for correctness; and a model selection reward that imposes a penalty of −0.15 for choosing a sub-optimal model (0 otherwise). The optimal model is pre-defined as Qwen2.5-Coder-7B-Instruct for MBPP, and identified by GPT-4o judging offline evaluations for Calculator and NQ.

The evaluation spans 9 tasks and 15 benchmarks. The candidate pool contains six heterogeneous open-source LLMs (Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, InternLM3-8B-Instruct, DeepSeek-R1-Distill-Qwen-7B, Qwen2.5-Coder-7B-Instruct, and the multi-modal Qwen3-8B-VL-Instruct) plus four foundation tools (Code Interpreter, Web Search, Calculator, Process Reward Model) and four multi-modal tools (Qwen3-Chart, Qwen3-Counting, Qwen3-Geo, Hunyuan-OCR). Benchmarks cover math (AIME2024, AIME2025, AMC), code (HumanEval, MBPP), arithmetic (Calculator), commonsense (NQ, WebQ), logic (LogiQA2), science (GPQA), and multi-modal tasks (ChartQA, Geometry3K, TallyQA, CountBench, TableVQA), with accuracy as the primary metric. The RL policy uses Qwen2.5-3B-Instruct, trained with batch size 32 for 250 steps at a learning rate of 1 × 10⁻⁶.

Why This Matters

Impact on research: The paper argues for a shift from model-centric scaling to ecosystem-centric orchestration, treating the pairing of a model with a tool as the unit of optimization rather than the model alone. It bridges two lines of work that had stayed separate: query-based LLM routing and RL for tool use.

Real-world applications:

• Multi-domain assistants that route math problems to a calculator or verification model, retrieval questions to web search, and code tasks to a code-specialized model.
• Diagram and document understanding, using chart extraction, OCR, counting, and geometry tools alongside a vision-language model.
• Cost-aware enterprise deployments, where the α parameter lets operators tune the accuracy-versus-expense trade-off and where simple retrieval queries consume few API calls.
• Evolving model marketplaces, since new specialist models and verifiers can be added to the routing pool at inference time without retraining.

Industry relevance: The results show that orchestration can let 7B and 8B open-source models approach or exceed GPT-4o on several benchmarks, which matters for organizations balancing capability against inference cost and for those building systems on top of fast-changing model and tool inventories.

Future Directions

• Extending beyond text and visual reasoning to other modalities such as audio and video, which the authors list as unexplored.
• Building robust fallback mechanisms and lightweight policy architectures, since the framework assumes reliable API access and may suffer from network latency or service unavailability in real deployments.
• Investigating how well transferable routing principles hold as the routing pool keeps growing, given that classifier-based baselines showed misaligned decision boundaries under pool extension.
• Further exploration of test-time scaling and model-tool preference analysis, which the paper flags as additional discussion deferred to its appendix.

Target Audience

Researchers and engineers working on LLM agents, model routing, tool-augmented reasoning, and reinforcement learning for language models will benefit most. Practitioners building multi-model, multi-tool production systems will find the cost-accuracy framing and the no-retraining pool extension result directly applicable. Readers without background in embedding-based clustering or policy-gradient RL will need to consult the cited routing and PPO literature first.

Authors’ abstract

The integration of large language models (LLMs) with external tools has significantly expanded the capabilities of AI agents. However, as the diversity of both LLMs and tools increases, selecting the optimal model-tool combination becomes a high-dimensional optimization challenge. Existing approaches often rely on a single model or fixed tool-calling logic, failing to exploit the performance variations across heterogeneous model-tool pairs. In this paper, we present ATLAS (Adaptive Tool-LLM Alignment and Synergistic Invocation), a dual-path framework for dynamic tool usage in cross-domain complex reasoning. ATLAS operates via a dual-path approach: (1) \textbf{training-free cluster-based routing} that exploits empirical priors for domain-specific alignment, and (2) \textbf{RL-based multi-step routing} that explores autonomous trajectories for out-of-distribution generalization. Extensive experiments across 15 benchmarks demonstrate that our method outperforms closed-source models like GPT-4o, surpassing existing routing methods on both in-distribution (+10.1%) and out-of-distribution (+13.1%) tasks. Furthermore, our framework shows significant gains in visual reasoning by orchestrating specialized multi-modal tools.