CureAgent: A Training-Free Executor-Analyst Framework for Clinical Reasoning

AI summary

Overview

Research area: Clinical decision support with LLM-based multi-agent systems; tool-augmented biomedical reasoning on the CURE-Bench benchmark (a NeurIPS 2025 competition).

Technical level: Advanced. The paper assumes familiarity with fine-tuned tool-calling agents, long-context foundation models, self-consistency sampling, ensemble topologies, and benchmark evaluation on open versus private leaderboards.

Scope: The paper describes a training-free, two-role agent architecture (a tool-executing "Executor" and a reasoning "Analyst") plus post-processing, and reports benchmark ablations over temperature, self-consistency sample counts, and ensemble topology.

What This Paper Is About

Small fine-tuned clinical agents such as TxAgent (a Llama-3.1-8B model with 8B parameters) are good at calling biomedical tools but frequently fail to actually use the evidence they retrieve when producing a diagnosis—a problem the authors name Context Utilization Failure. The paper's goal is to close that gap without expensive end-to-end finetuning, by splitting the job into a specialized retrieval agent and a separate long-context reasoning model that synthesizes the retrieved evidence.

Key Contributions

1. The Executor–Analyst Framework: A modular architecture that decouples tool-execution syntax (handled by fine-tuned TxAgent Executors) from clinical reasoning and evidence synthesis (handled by long-context Gemini 2.5 Analyst models), with a deterministic post-processing layer for format calibration and deduplication.
2. A Stratified Ensemble (Late Fusion) topology: Compared against a Global Pooling (Early Fusion) baseline, the stratified design partitions Executor instances into parallel subgroups so diverse retrieval paths survive until a final answer-level self-consistency vote, avoiding an early information bottleneck.
3. A failure-mode analysis of the baseline agent: On the validation set, 73 of 413 multi-choice questions failed, with Reasoning & Retrieval Failure at 65.8%, Output Parsing Errors at 19.2%, and Instruction Adherence Failures at 12.3%.
4. Two stress-test findings framed as scaling insights: a Context-Performance Paradox (accuracy degrades when reasoning contexts exceed 12,000 tokens) and a Curse of Dimensionality in action spaces (accuracy drops when the tool library grows from 200+ to 600+ tools).

Main Findings

• Decoupling is the largest single gain. A single TxAgent Executor paired with a single Gemini 2.5 Flash Analyst reached 74.701% on phase2, versus 69.325% for standalone TxAgent and 63.104% for standalone Gemini 2.5 Flash.
• Topology matters more than raw pool size. Config A (Global Pooling) with n1=30 and n2=3 reached 80.510%. Config B (Stratified Ensemble) with n1=10 and n2=3 reached 81.367%, which the paper calls state-of-the-art for its non-search configuration.
• Search adds further gains. The leaderboard configuration, Config B with n1=10 Executors and n2=3 Gemini 2.5 Flash Analysts equipped with search, reached 83.803% on phase2.
• Self-consistency helps with diminishing returns. TxAgent's greedy decoding on phase2 was fixed at 69.3%. Self-consistency rose to about 73.3% in the low-sample regime (n<15), plateaued beyond n>20, and reached approximately 74.2% at n=60.
• Executor temperature needs calibration. On phase1, TxAgent scored 58.950 at temperature 0.6, 59.248 at 0.7, 65.214 at 0.8 (the peak, and the value fixed for later experiments), and dropped to 56.747 at 0.9.
• Fine-tuned tool use beats general reasoning in zero-shot settings. On phase2, TxAgent scored 69.325 while open-source general models scored far lower (for example Llama-3.1-8B at 29.715, Qwen3-8B at 36.264, gpt-oss-120b at 39.896), and specialized medical models such as Llama3-Med42-8B (50.407) and medgemma-4b-it (54.366) still lagged well behind.
• Closed-source reasoning models benefit strongly from generic search. Gemini 2.5 Flash moved from 63.104 without tools to 69.627 with search; Gemini 2.5 Pro from 67.753 to 74.806; Gemini 3 Pro Preview from 70.985 to 81.283.
• Larger tool libraries hurt retrieval. On the validation set, TxAgent scored 92.0% with ToolUniverse 1.0 (200+ tools) but 87.5% with ToolUniverse 2.0 (600+ tools).
• Longer contexts hurt reasoning. Accuracy fell from 94% to 87.93% when reasoning contexts exceeded 12,000 tokens.
• The approach requires no finetuning. All gains came from architectural composition of existing models plus a deterministic post-processing module.

Methodology in Plain English

The authors first diagnosed why the competition baseline agent fails, by manually categorizing 73 failed validation questions into error types. The dominant category was not an inability to find evidence but an inability to use it. They then tested whether simply swapping in stronger open-source or closed-source models would fix the problem, and found that neither general models nor specialized medical models matched the fine-tuned TxAgent on this tool-centric benchmark, while strong closed-source reasoning models still benefited substantially from having search.

Based on that, they built a two-part pipeline. The Executor is the existing fine-tuned TxAgent; it does not answer the question, it only retrieves. To reduce the randomness of a single retrieval run, they instantiate multiple Executor copies in parallel and keep the most frequently selected tool calls and the most frequent reasoning trace. The Analyst is a Gemini 2.5 model that receives this noisy aggregated evidence, cross-references it, optionally performs supplementary web searches, and writes a chain-of-thought rationale and a draft answer. A final lightweight deterministic module uses regular expressions to force the draft into the benchmark's required answer format and removes duplicate responses to the same query.

The key architectural experiment compares two ways of organizing the same compute budget. In Global Pooling, all Executor runs are merged into one context before any reasoning begins, then several Analysts reason over it. In the Stratified Ensemble, Executor runs are split into parallel subgroups, each subgroup produces its own context and its own Analyst chain, and consensus is taken only at the level of final answers. The stratified version performed better, which the authors attribute to preserving minority but clinically important evidence that early pooling would discard.

Why This Matters

Impact on research. The paper argues that architectural engineering—interaction design, ensemble topology, output calibration—can recover substantial performance without parameter updates, and that the bottleneck in clinical agents is often context utilization rather than retrieval capability. It also documents two scaling problems (context length and tool-library size) that any future clinical agent design will face, and shows that generic web search and task-specific tool execution are complementary rather than competing.

Real-world applications:

• Clinical decision support systems that must ground recommendations in frequently updated sources such as FDA labels, Open Targets, and the Human Phenotype Ontology.
• Drug interaction and contraindication checking, where the Executor's precision matters and the Analyst's synthesis checks retrieved evidence against a patient's specific comorbidities.
• Systems that must integrate new biomedical APIs or databases without retraining a model each time a tool is updated.
• Deployment scenarios where opaque or hallucinated outputs are unacceptable, addressed here via deterministic format calibration and deduplication for reliability.

Industry relevance. Because the framework is training-free, organizations can upgrade the reasoning backbone or expand the tool library independently, which lowers the cost of keeping a clinical agent current. The results also show a practical tradeoff: adding tools improves coverage but degrades retrieval accuracy, so commercial deployments with large API catalogs need hierarchical or retrieval-based tool selection rather than flat tool lists.

Future Directions

• Hierarchical tool indexing. Organize tools into clinical taxonomies so retrieval can proceed coarse-to-fine instead of searching a flat space, addressing the drop from 92.0% to 87.5% when moving from 200+ to 600+ tools.
• Training-free adaptation to massive toolsets. The authors propose In-Context Learning or RAG-based documentation retrieval to navigate tool libraries of more than 1,000 tools without finetuning for every API update.
• Information compression and early rejection. Rather than feeding all aggregated evidence into the context, implement confidence-based filtering (referencing DeepConf) that estimates semantic redundancy and rejects irrelevant evidence before it consumes the context window—motivated by the decline from 94% to 87.93% beyond 12,000 tokens.
• Redrawing the Executor–Analyst boundary. As models such as Gemini 3 Pro become capable of stronger zero-shot tool use, the specialized fine-tuned Executor could evolve into a lightweight prompt-engineered module, though the authors state the decoupling principle remains essential.

Target Audience

This paper is most useful to researchers and engineers working on tool-augmented LLM agents, clinical decision support, and multi-agent orchestration—particularly those evaluating on CURE-Bench or similar tool-retrieval benchmarks. It is also relevant to practitioners deciding between finetuning a small domain model and composing a fine-tuned retriever with a strong long-context reasoning model, and to competition participants interested in the winning solution to CURE-Bench@NeurIPS 2025. Readers without background in agent topologies or LLM evaluation will find the methodology section accessible but the benchmark tables harder to interpret without context.

Authors’ abstract

Current clinical agent built on small LLMs, such as TxAgent suffer from a \textit{Context Utilization Failure}, where models successfully retrieve biomedical evidence due to supervised finetuning but fail to ground their diagnosis in that information. In this work, we propose the Executor-Analyst Framework, a modular architecture that decouples the syntactic precision of tool execution from the semantic robustness of clinical reasoning. By orchestrating specialized TxAgents (Executors) with long-context foundation models (Analysts), we mitigate the reasoning deficits observed in monolithic models. Beyond simple modularity, we demonstrate that a Stratified Ensemble strategy significantly outperforms global pooling by preserving evidentiary diversity, effectively addressing the information bottleneck. Furthermore, our stress tests reveal critical scaling insights: (1) a \textit{Context-Performance Paradox}, where extending reasoning contexts beyond 12k tokens introduces noise that degrades accuracy; and (2) the \textit{Curse of Dimensionality} in action spaces, where expanding toolsets necessitates hierarchical retrieval strategies. Crucially, our approach underscores the potential of training-free architectural engineering, achieving state-of-the-art performance on CURE-Bench without the need for expensive end-to-end finetuning. This provides a scalable, agile foundation for the next generation of trustworthy AI-driven therapeutics. Code has been released on https://github.com/June01/CureAgent.