ToolScope: Enhancing LLM Agent Tool Use through Tool Merging and Context-Aware Filtering

AI summary

Overview

Research area: Natural Language Processing / LLM agents, specifically tool selection and retrieval for agentic systems.

Technical level: Intermediate — readers should be comfortable with retrieval concepts (sparse vs. dense retrieval, reranking, embeddings) and basic graph terminology, but the paper's framing is applied and engineering-oriented.

Scope: The paper introduces ToolScope, a two-part framework (ToolScopeMerger and ToolScopeRetriever) that reduces semantic redundancy across large agent toolsets and compresses the number of tools passed to an LLM, evaluated on 3 benchmarks with 3 LLMs.

What This Paper Is About

LLM agents depend on external tools, but real-world toolsets contain many tools with overlapping names and descriptions, which creates ambiguity and hurts the agent's ability to pick the right tool. At the same time, LLM input windows are limited, so an agent cannot efficiently consider hundreds or thousands of tools per query.

ToolScope attacks both problems at once: it automatically merges semantically redundant tools into consolidated ones (with an LLM-based Auto-Correction step to catch bad merges), and it retrieves only the top-k most relevant tools per query so the toolset fits inside the model's context window.

Key Contributions

1. ToolScopeMerger — a graph-based, automated and scalable framework with Auto-Correction that merges semantically similar tools to reduce overlap in large toolsets, without requiring manual curation.
2. ToolScopeRetriever — a hybrid retrieval system that combines sparse and dense scores with query decomposition, cross-encoder reranking, and min-max score normalization to reduce context length while preserving selection accuracy.
3. A joint framework combining merging and retrieval that improves retrieval and selection accuracy across open-source benchmarks and models, boosting tool selection accuracy by 34.6% on Seal-Tools, 38.6% on UltraTool, and 8.8% on BFCL.
4. Diagnosis of benchmark overlap — the paper reports that publicly available tool-use benchmarks (Seal-Tools, BFCL, UltraTool) contain semantically redundant tools that degrade retrieval and selection, limiting the reliability of benchmarking research.

Main Findings

• State-of-the-art correct selection rate: ToolScope, with or without Auto-Correction, achieves the highest CSR@k on Seal-Tools, BFCL, and UltraTool across all evaluated k values and all three LLMs (GPT-4o, LLaMA-3.3-70B, Command-R-08-2024). The abstract reports gains ranging from 8.38% to 38.6%.
• Gains vary by benchmark: Large gains on Seal-Tools (+34.6%) and UltraTool (+38.6%), smaller but consistent gains on BFCL (+8.8%). GPT-4o achieves the highest CSR among the three models.
• Auto-Correction helps, but not universally: It yields 1.5–2.9% gains on Seal-Tools and 2.9–7.9% on UltraTool. On BFCL the gains are modest for GPT-4o (+2.6%) and LLaMA-3.3-70B (+5.0%), with a slight regression for Command-R (-1.2%), which the authors attribute to over-correction when baseline coverage is already strong.
• Merging is the dominant contributor: In the ablation, enabling Merger with the Reranker off boosts CSR by 22.0% on Seal-Tools, 5.0% on BFCL, and 7.0% on UltraTool.
• Reranking helps mainly on large, noisy toolsets: Gains of +1.3% on Seal-Tools and +0.9% on UltraTool, but 0.0% on BFCL, which is a single-tool benchmark.
• Context length drops sharply: Average total context length falls from 32,563 to 469 tokens on BFCL (98.6% reduction), from 292,107 to 317 on Seal-Tools (99.9%), and from 136,352 to 2,076 on UltraTool (98.5%).
• Toolset size shrinks after merging: BFCL goes from 400 to 344 tools (-14.0%), Seal-Tools from 4076 to 3992 (-2.1%), and UltraTool from 1885 to 1408 (-25.3%).
• Functionality is largely preserved: Tool-Call Coverage Rate and Unique Capability Coverage show 82–95% call-level coverage and 80–96% capability-level coverage. For infrequent capabilities (≤3 occurrences), retention is 82.0% (BFCL), 95.8% (Seal-Tools), and 86.4% (UltraTool).
• Merge quality validated by humans: Human evaluation on 48 BFCL tool clusters shows 95.4% correct merge decisions; Auto-Correction achieves 94.4% F1 (95.5% precision, 93.3% recall).
• Retrieval improves: Recall@10 rises from 0.550 to 0.935 on Seal-Tools and from 0.945 to 0.985 on BFCL.
• Dense-only retrieval wins: α tuning shows retrieval@k peaks at α = 1, meaning dense-only retrieval performs best; the final configuration reranks the top-50 candidates with a cross-encoder using min–max normalization.
• Robust to documentation quality: ToolScope maintains CSR of 85.7% (BFCL), 95.0% (Seal-Tools), and 72.7% (UltraTool) even on low-quality documentation, despite Seal-Tools and UltraTool containing 79% and 96% medium- or low-quality tools.
• Preliminary end-to-end result: On a small-scale 20-query full tool-calling evaluation, ToolScope reaches 80% final answer accuracy versus 30% for BM25 and 40% for dense retrieval.
• Threshold robustness: Cosine similarity threshold sensitivity on Seal-Tools swept 0.77 to 0.90; CSR is stable from 0.77 to 0.82 (three lowest thresholds differ by less than 0.5%), peaks at 0.82, and declines above 0.86. The paper fixes the threshold at 0.82.

Methodology in Plain English

ToolScope has two halves.

ToolScopeMerger treats the toolset as a graph problem in three stages. First, every tool's signature and description is embedded (using the thenlper/gte-large model) and indexed in FAISS; each tool's top-k most similar neighbors by cosine similarity become merge candidates, with a similarity threshold of 0.82. Second, an LLM-based binary classifier (GPT-4o) judges each candidate pair and answers whether the two tools serve sufficiently similar functions to justify merging; pairs classified as equivalent become edges in an undirected "pruning graph." Connected components in that graph form clusters of overlapping tools. Third, one representative tool per cluster is kept (the implementation picks the tool with the shortest function name string length), and an LLM synthesizes a new unified signature and description for the merged tool. An optional Auto-Correction module uses an LLM validator to audit each cluster: if the cluster is valid the merge proceeds; if not, the validator splits it into finer sub-clusters and removes non-equivalent members. Finally, the benchmark's gold labels are relabeled through the mapping φ so the new toolset remains compatible with the evaluation.

ToolScopeRetriever handles selection. For single-tool queries, it computes a hybrid score as a weighted average of dense and sparse (BM25) similarity, takes the top M candidates, and reranks them with a cross-encoder. For multi-tool queries, each subquery goes through the same procedure; the top-1 tool per subquery is kept, and remaining candidates are min-max normalized so scores from different subqueries are comparable, then added iteratively until k tools are selected.

Evaluation uses 3 benchmarks (BFCL, Seal-Tools, UltraTool) covering 400, 654, and 4814 queries and 400, 4076, and 1885 tools respectively, with 3 LLMs as the selection agents. Baselines are BM25, Dense embeddings, and (in the appendix) DPR and ToolShed. The primary metric is Correct Selection Rate (CSR@k), with Recall@k for retrieval.

Why This Matters

Impact on research: The paper reframes tool selection as two coupled problems — toolset redundancy and context budget — that prior work treated separately. It shows that hybrid retrieval had not been combined with automated tool merging before, and it flags that popular open-source benchmarks contain semantically overlapping tools that distort evaluation, which is a methodological caution for the field.

Real-world applications:

• Enterprise agent platforms with large internal API catalogs, where tool names and descriptions drift and duplicate over time.
• Customer-facing assistants that must pick the correct service action from hundreds of registered functions under tight prompt budgets.
• Rapidly evolving developer toolchains or MCP-style tool servers where new tools are added continuously and manual curation does not scale.
• Cost-sensitive deployments where shrinking tool context lowers token usage — the paper measures reductions as high as 99.9%.

Industry relevance: The work comes from Oracle AI and targets production settings: the retriever is evaluated on the full toolset rather than a pre-filtered one, which the authors describe as reflecting production usage. ToolScopeMerger is also reported not to be tied to GPT-4o — open-source alternatives (LLaMA 3.3 70B and LLaMA 3.1 8B) yield comparable performance — which matters for organizations that cannot send tool metadata to a closed model.

Caveat the authors state: ToolScope is not yet suitable for deployment in domains where tool selection errors carry significant consequences, such as medical, legal, or financial fields, because the merging and retrieval mechanisms are probabilistic and subject to hallucination, bias, and misclassification.

Future Directions

• Richer tool metadata: The current retrieval relies only on tool names, signatures, and descriptions. Adding typical use cases, domains, and input/output schemas could strengthen ranking, especially for very large toolsets.
• Multi-index retrieval: The authors propose a more extensible multi-index framework to improve scalability and relevance.
• Extending evaluation beyond selection: Future work would analyze ToolScope's impact on tool calling and response generation across multiple datasets to establish a holistic picture of improvements to agent tool learning.
• Reasoning-oriented models: The paper notes that evaluating with reasoning-oriented models could affect the effectiveness of Auto-Correction and the quality of tool selection.

Target Audience

Researchers and engineers working on LLM agents, tool learning, and agentic retrieval systems will get the most from this paper, particularly those building or maintaining large production toolsets. It is also relevant to benchmark designers, since it documents a pervasive tool-overlap problem in existing tool-use datasets, and to applied NLP practitioners weighing the tradeoffs between automated merging, hybrid retrieval, and reranking under strict context budgets.

Authors’ abstract

Large language model (LLM) agents rely on external tools to solve complex tasks, but real-world toolsets often contain redundant tools with overlapping names and descriptions, introducing ambiguity and reducing selection accuracy. LLMs also face strict input context limits, preventing efficient consideration of large toolsets. To address these challenges, we propose ToolScope, which includes: (1) ToolScopeMerger with Auto-Correction to automatically audit and fix tool merges, reducing redundancy, and (2) ToolScopeRetriever to rank and select only the most relevant tools for each query, compressing toolsets to fit within context limits without sacrificing accuracy. Evaluations on three state-of-the-art LLMs and three open-source tool-use benchmarks show gains of 8.38% to 38.6% in tool selection accuracy, demonstrating ToolScope's effectiveness in enhancing LLM tool use.