Tool Discovery, Selection, and Routing

Tool definitions were consuming 134K tokens before optimization. Fifty-eight tools drawn from common MCP servers take about 55K tokens of context before a conversation has started. Anthropic reported both numbers in November 2025, from production rather than from a hypothetical.

The errors that follow are not spread evenly across the catalogue. “The most common failures are wrong tool selection and incorrect parameters, especially when tools have similar names like notification-send-user vs. notification-send-channel.” The pair is the whole problem in two strings. Both are relevant to the same request. Nothing in the wording separates them. The model has been handed both.

Defer the definitions and discover them on demand, and a 50+ MCP-tool setup drops from about 77K tokens of context to about 8.7K — an 85% reduction. Accuracy rose as the context fell. MCP-evaluation accuracy went from 49% to 74% for Opus 4, and from 79.5% to 88.1% for Opus 4.5. The same catalogue, differently exposed, is a different system.

Tool selection is a retrieval and policy problem as well as a reasoning problem. Large catalogs raise context cost, invite naming collisions, and increase the chance of choosing a plausible but unsuitable capability. A runtime can filter tools by task state, tenant, role, resource, and risk. Dynamic discovery should return a small, authorized set rather than every registered function.

This is no longer only a design opinion. The Model Context Protocol specification, revision 2025-11-25, writes it into normative text. Per-client consent is a MUST: “To prevent confused deputy attacks, MCP proxy servers MUST implement per-client consent and proper security controls as detailed below.” On identity it is equally flat: "MCP servers MUST NOT accept any tokens that were not explicitly issued for the MCP server". Its Scope Minimization section tells servers to "Emit precise scope challenges; avoid returning the full catalog". Two habits are listed there under Common Mistakes: "Publishing all possible scopes in scopes_supported" and "Returning entire scope catalog in every challenge".

A team that ships the full catalog on every call is not choosing a simpler architecture. It is shipping two of the specification's named common mistakes, in a revision that can be cited by number.

Routing begins at the capability registry, which stores tool identity, version, owner, schema, and lifecycle state. An eligibility filter then removes tools outside the current user, task, or environment scope. Semantic retrieval finds candidates whose descriptions match the requested operation. Policy routing applies risk, cost, latency, and regional constraints. Only then does model selection choose among the bounded candidates, or ask for clarification. Policy routing and model selection need separate owners and separate tests.

The registry's version and owner fields look like bookkeeping until a tool changes underneath them. The npm package postmark-mcp was published on 15 September 2025 by the handle phanpak, a copy of Postmark's (ActiveCampaign's) legitimate MCP server. It behaved correctly through versions 1.0.0–1.0.15. Version 1.0.16 added a single line, on line 231, that BCC'd every outgoing email to phan@giftshop.club. Koi Security's Idan Dardikman put the trust-building phase plainly: “For 15 versions - FIFTEEN - the tool worked flawlessly. Developers were recommending it to their teams.”

Koi Security disclosed it on 25 September 2025 as the first malicious MCP server found in the wild. Roughly 1,500 downloads a week, 1,643 in total, an estimated 300 active organisations, and 3,000–15,000 exfiltrated emails a day. The developer deleted the package rather than explaining it. The name, the schema and the description never changed. Only the version did. A registry that does not pin and attribute versions has nothing to detect.

Three catalogs are on offer, and they differ in what reaches the model at all. The static full catalog supplies every tool on every model call: simple to prototype, high in context cost, weak on lifecycle control. The state-scoped catalog exposes only the tools valid for the current workflow state. That buys a predictable action space and a strong policy boundary, at the price of designing the states. The retrieved catalog discovers capabilities from a registry at runtime. It scales to large ecosystems, it must enforce authorization, and it needs retrieval evals.

That last requirement has a measured ceiling behind it. ToolRet is a tool-retrieval benchmark published in 2025: 7.6k retrieval tasks over a corpus of 43k tools merged from 34 datasets. Its headline result: “Our results reveal that even the best model (i.e., NV-embedd-v1) that demonstrates strong performance in conventional IR benchmarks, achieves an nDCG@10 of only 33.83 in our benchmark.” NV-Embed-v1 is an NVIDIA model of about 7.85B parameters, built on Mistral-7B. It is not an outlier in the wrong direction. Every retriever in the experiments scored under 35% on Completeness@10 and under 52% on recall@10.

Whichever catalog a team picks, the question asked of it is the same. For a given request, is the correct tool present in the authorized candidate set? On 43k tools, a strong general-purpose retriever misses it more often than not. And each catalog fails more quietly than it succeeds. A catalog that ranks by similarity will hand back a tool the caller has no permission to use, and the score attached to it says nothing at all about permission.

A routing benchmark needs a real catalog, paraphrased requests, and at least one deprecated near-twin. Build one and run it. Collect real intents, including paraphrases, ambiguous requests, and unsupported actions. Define the eligible sets, recording which tools are permitted for each task state and identity. Test retrieval recall, so the correct capability appears without exposing forbidden ones. Measure final selection separately: that is what separates discovery error from model-choice error. Then exercise lifecycle changes — retire, rename, and version tools without silently changing behavior.

Build the unsupported-action case first. The failure there is the largest one measured anywhere in this lesson. MetaTool built the ToolE dataset of 21,127 user queries — 20,630 single-tool and 497 multi-tool. It then tested tool selection with the correct tool deliberately removed from the candidate list. The average model correct-selection rate on that sub-task was 8.59%. Humans reached 96%: “Notably, in sub-task 3, human performance reaches an impressive 96%, a stark contrast to the model’s meager 9%.” Most models stayed below 20% even with five-shot prompting.

A person handed a list without the right tool on it says so. A model, nine times in ten, calls something else. Keep, for each request, the list of tools that were offered and the one that was called. That pair is what a reviewer will ask to see, and it is the only record that distinguishes the two.

Tool routing is easiest to debug when registry, eligibility, retrieval, and final selection are logged as separate stages. The log is what makes a review of tool discovery and routing possible. It shows which tools existed, which of them the caller was allowed to use, which ones the retriever surfaced, and which one the model finally chose.

Each of the failures in this lesson lands in a different one of those four. postmark-mcp is a registry failure: identity, schema and description unchanged, behaviour changed at version 1.0.16. The Tymshare compiler is an eligibility failure: the authority checked was the deputy's, not the caller's. ToolRet's nDCG@10 of 33.83 over 43k tools is a retrieval failure, with the right tool never reaching the candidate set. MetaTool's 8.59% against 96% is a selection failure, with the model choosing from a list that could not contain a right answer. Four causes, four owners, four fixes.

Collapse those four into a single line and every bad call looks alike. The mistake could have been made at any one of the stages, and each stage is fixed by a different person.