Route Anything: A Study of Model and Policy Routing for General-Purpose AI Systems

AI summary

Overview

Research area: Machine learning systems — model routing, model portfolios, and multi-model orchestration spanning language, image, video, speech, code agents, and robotics.

Technical level: Advanced. The paper pairs a formal routing theory (propositions, a theorem, a certificate) with a large empirical study, so readers need comfort with expected value, cost-aware optimization, and benchmark evaluation.

Scope (one sentence): The paper builds a unified theory and an omni-modal evaluation matrix (25 benchmarks, 374,401 scored runs, 452 candidate configurations) to measure how much a per-input router can gain over the best single model, how well learned routers realize that gain, and when a cheaper route is good enough.

What This Paper Is About

A general-purpose AI system can draw on many specialized models, tools, and policies — a language model for questions, a generator for images or video, a speech model, a learned policy for a robot — and the best choice changes from one input to the next. The paper asks how much a system gains by routing each input to its best-suited candidate instead of always calling one fixed model, and how much of that gain a learned router can actually capture.

It also asks when routing pays off in cost: whether a cheaper route can match a strong reference model's quality while cutting price or inference time, and under what conditions that guarantee holds on unseen inputs rather than only on the data used to fit the router.

Key Contributions

1. A unified routing theory. The authors formalize routing as choosing, in each observed state, the candidate maximizing expected value minus a priced cost. Lemma 1 gives a reference-relative efficiency formula, Theorem 2 characterizes optimal routing at a price and under a quality floor, Proposition 3 gives an "information ladder" bounding what a router can reach from coarse states (benchmark or task type) up to the per-input oracle, Proposition 4 argues for regressing outcomes rather than classifying winners, and Proposition 5 supplies a certified quality floor with a finite-sample test along a "price path."

2. An omni-modal trajectory matrix. 452 candidate configurations run on the same inputs across 25 benchmarks in seven groups — text QA, image-text QA, image generation, video generation, speech generation, code and workplace agents, and robot manipulation — producing 374,401 scored runs.

3. Quantified potential and realized gains from routing. The per-input oracle beats the best fixed candidate by 5.4 points on 19,814 test inputs from 24 benchmarks; learned routers beat the fixed candidate by 0.3 points across 16 benchmarks, 2.1 points on video generation, and 1.1 points on robot manipulation with 24% less policy time.

4. A cost analysis of when routing improves efficiency. Routing raises efficiency over the reference by 8.82× on average across four testbeds, including 25.5× on code and 7.1× on text QA, plus evidence that richer candidates and finer states are worth buying only when their value net of full cost beats what they replace.

Main Findings

• The potential gain is large and sits within task types. On 19,814 test inputs the best fixed candidate scores 82.60 and the per-input oracle 87.98, a 5.38-point gap. It ranges from 0.68 points on speech generation to 18.93 on video generation and 18.40 on robot manipulation. Picking the best candidate per benchmark in hindsight moves 82.60 only to 82.62, and the best candidate per capability leaf, also in hindsight, closes 8% of the gap on 18 benchmarks (77.89 against 77.03 fixed and 87.44 oracle).

• Learned routers produce modest but real gains. One capability-tree router over 16 benchmarks scores 79.98 against 79.68 for the fixed candidate and 84.80 for the oracle — a 6% share of the oracle's advantage. Gains concentrate in Terminal-Bench (+8.3 on 36 inputs), VBench-2.0 (+2.1), OneIG (+1.1), and GDP.pdf (+1.0), with the largest loss on AA-LCR (−2.4). On GenEval2 and Qwen-Image-Bench, none of 124 router configurations across 31 designs beats the fixed generator with an interval above zero, and 44 are significantly worse.

• On robots, the router reads the scene and saves time. Given the instruction, three 240×320 camera frames, and robot state, it sends 929 of 1,000 clean scenes to PatchWAM and 842 of 1,000 randomized scenes to ME-Dex. Overall success is 71.55% against 70.45% for the best single policy (+1.10, 95% CI [−0.40, +2.55]) at 11.05 instead of 14.51 seconds of policy time per scene — 24% less, or 1.33× the success per policy-second.

• Learn outcomes, not winners. With identical inputs and backbone, per-policy outcome regression reaches 71.6% robot success while a winner-label classifier reaches 42.8%, because a latency tie-break decides 94% of winner labels and 24,556 of 31,408 successful training outcomes become wrong-class targets. The best of 12 generative winner-label recipes scores 78.87 against 79.44 for the fixed candidate on 16 benchmarks.

• Compare candidates on the same inputs. A fixed baseline requiring a score on every training input dropped Opus 5.5, which missed 11 of 1,122 HLE scores; HLE fell from 58.24 to 52.90 and AA-Omniscience from 77.50 to 70.00, dropping the pooled baseline from 68.30 to 65.52 and turning the outcome router's 0.40-point deficit into a 2.38-point gain. Matching portfolios also matters: comparing both on the same tasks picks the best added model in 96.4–97.1% of CodeRouterBench comparisons, versus 86.3–94.3% for averages frozen at the current portfolio.

• A cheaper route is often good enough. Efficiency rises 8.82× on average over the reference across four testbeds. On CodeRouterBench, Opus 4.6's admission price is λ_B = 100 score per dollar (2.9 η_B), below the zero-loss floor price of 144 and far below η* = 1022, so the zero-loss router never calls it and reaches 25.5× at a 0.44-point loss. Always calling Qwen3-Max already gives 20.1× at a 4.2-point loss; state-based routing recovers that loss while adding 27% efficiency.

• A fitted floor is not a guarantee, but a certified one is. Refitted on half the probing tasks, the zero-loss CodeRouterBench floor holds on test in only 12.5% of 200 splits; certified with Proposition 5 it holds in all 200, at a median 19.4×.

• Richer candidates and finer states must beat what they replace. Across all 42 ordered model pairs, the second model succeeds less often after the first fails (median ratio 0.49). A free compiler gate gives 4.09× dollar efficiency at a 2.4-point loss, while an LLM judge gives 2.63× if free but 0.30× at list price; in seconds, always-Opus beats every verified cascade. A DeepSeek-v4-pro forecast costs 2.4 attempts at list price, while a prompt-only TF-IDF router costs 0.06% of one.

Methodology in Plain English

The authors start with theory: a router sees some state (just the benchmark name, a task type, or the full input with images) and picks a candidate, earning that candidate's benchmark score minus a price times its cost. They prove that the best thing a router can do in each state is pick the candidate with the highest value-minus-priced-cost, that better information can only raise the attainable score up to a ceiling set by unpredictable outcome noise, and that maximizing efficiency under a quality floor is equivalent to routing at one specific price set by the reference and the tolerance.

To test this, they run every candidate in a pool on exactly the same inputs, so all comparisons are paired. The pools mix frontier and cheaper language models, image, video and speech generators, and seven vision-language-action robot policies. Inputs are split 70/10/20 into training, development, and test by source group, and RoboTwin's 2,000 test scenes come from 10 tasks never seen in training. They then measure three rungs of the ladder: the best fixed candidate, learned routers, and the per-input oracle.

Their routers fine-tune Qwen3.5-4B on the task text and output requirement — plus camera frames and robot state for robots — and either regress each candidate's expected outcome or classify the input into a capability type and look up that type's best candidate. For cost experiments they add two testbeds with recorded token prices (CodeRouterBench and RouterBench) plus recorded prices for image and speech generation and inference time for robots, fit routes on probing data, and evaluate on held-out test tasks. Finally, they replace the fitted floor with a statistical certificate: testing routers in order along the price path and returning the reference if nothing passes.

Why This Matters

The paper shifts routing research from language models alone to a single matrix spanning text, images, video, speech, code agents, and robots, and it supplies a theory that tells practitioners why a router's gain is bounded — by what it observes and by irreducible outcome noise. The certified-floor result is a practical check against the common failure mode where a router meets its quality target on fitting data but silently misses it in deployment.

Real-world applications:

• Production AI assistants and API gateways that must answer text, code, and image requests under a fixed budget while holding quality within a tolerance of a strong reference.
• Cost-optimized coding agents, where the paper shows 25.5× efficiency at a 0.44-point loss by never calling the expensive reference.
• Robot fleets where a lightweight router reads camera frames to pick a policy per scene condition, cutting per-scene policy time by 24% without losing success.
• Generative media pipelines where a half-point quality tolerance cuts image generation cost by 26% by moving benchmarks to cheaper generators like Grok Imagine 2.0 and Qwen-Image 3.

Industry relevance: Router overhead is small relative to the resources it allocates — a 4B router takes 0.13s per decision on one H20 GPU, versus 27–158s to generate an image and 211–314s to generate a video. The findings on training targets (regress outcomes, not winners) and on matched comparisons give teams concrete, cheap-to-adopt rules for building router training sets that do not silently favor fast or fully-covered models.

Future Directions

• Closing the remaining gap to the oracle. Even the best learned router captures only 6% of the oracle's advantage, and 17% of the gap on RoboTwin requires predicting which policy succeeds on a given scene rather than a scene condition — an open perception-and-prediction problem.
• Making routers work where they currently fail. On GenEval2 and Qwen-Image-Bench none of 124 configurations beats the fixed generator, and on text QA the capability-tree router is 0.17 points below it; understanding when routing cannot help is unresolved.
• Cheaper and better routing signals. The paper shows forecasts can cost more than the attempt they replace and that a prompt-only TF-IDF router costs a tiny fraction of a model-based one, suggesting a search for signals whose value reliably exceeds their price.
• Extending the certificate beyond the studied testbeds. Proposition 5 is demonstrated on CodeRouterBench and RouterBench with specific p-value constructions; applying certified floors to video, speech, and embodied settings with their own cost structures is left open.

Target Audience

Researchers and engineers working on model routing, model cascades, LLM serving infrastructure, and multi-model orchestration will get the most from this paper, as will benchmark designers interested in paired, same-input evaluation and in the "compare candidates on the same inputs" baseline rule. Robotics and multimodal-generation practitioners will find the scene-conditioned routing and per-modality cost results directly applicable, while readers without a background in cost-aware optimization should expect the theory sections to require careful reading.

Authors’ abstract

Artificial general intelligence (AGI) requires a method that can understand, generate, reason, and act across diverse domains and modalities. Rather than relying on a single model to master every capability, a promising path toward AGI now is to orchestrate a heterogeneous ecosystem of specialized models, tools, and policies (candidates) into a unified AI system. At the heart of this vision lies a fundamental question: how can an AI system dynamically select and coordinate the most capable candidate for each task while balancing performance and computational cost? In this work, we present Route Anything, a unified framework for studying model and policy routing across general-purpose AI systems. We develop a unified theoretical foundation that characterizes optimal routing under quality--cost trade-offs and establishes how the information available to a router determines its attainable performance. To systematically investigate routing across diverse capabilities, we first construct an omni-modal trajectory matrix spanning 25 benchmarks across language understanding, image, video, and speech generation, coding and workplace agents, and robotic manipulation, comprising 374,401 scored runs from 452 candidate configurations. Building on this framework, we train and test different routers to investigate the potential of omni-modal model routing, the capabilities of learned routers, and the conditions under which routing improves efficiency. Our results reveal substantial opportunities for capability orchestration: per-input oracle routing outperforms the best fixed candidate by 5.4 points, and learned routers outperform it by 0.3 points across 16 benchmarks, by 2.1 points on video generation, and by 1.1 points on robot manipulation while using 24% less policy time. More importantly, routing raises efficiency by 8.82x on average compared with the reference model.