From Given to Gathered Evidence: Agentic Learning for Longitudinal Medical Reasoning

AI summary

Overview

Research area: Medical multimodal agents and agentic post-training (computer vision / vision–language models applied to longitudinal clinical reasoning).

Technical level: Advanced. The paper combines a UK Biobank–grounded multimodal benchmark, tool-use agent harnesses, rejection-sampling fine-tuning, Group Relative Policy Optimization (GRPO), privileged on-policy self-distillation (OPSD), and rubric-based LLM feedback (RLAIF).

Scope (one sentence): The paper argues that longitudinal medical reasoning should be evaluated and learned as evidence seeking rather than question–answer imitation, and it introduces the CASE benchmark, three role-specific clinical agents, and a two-stage post-training recipe for a compact 8B vision–language policy.

What This Paper Is About

Conventional medical benchmarks ask a model to reason over evidence that has already been selected and handed to it, whereas real longitudinal clinical reasoning requires deciding which records, visits, organs, sequences, and slices to inspect before answering. The authors build an interactive benchmark where the agent must autonomously gather the evidence that determines the answer from a patient-specific environment containing clinical context plus baseline and follow-up multi-sequence MRI. Their goal is to show that a compact, open vision–language model can learn a transferable investigation policy — learning both what evidence to seek and how to reason from what it finds — instead of merely imitating a stronger teacher's answer.

Key Contributions

1. Reframing longitudinal medical reasoning as evidence seeking. Reasoning is recast as an interactive problem in which the agent must determine what patient evidence to acquire before concluding, moving beyond fixed-context evaluation.

2. CASE: Clinical Agents for Seeking Evidence. The paper operationalizes this view with role-specific clinical agents, a patient-level tool-use harness, and a longitudinal multimodal benchmark derived from real ICD-10-coded diagnoses, requiring agents to navigate heterogeneous evidence and invoke specialist tools.

3. A longitudinal multimodal benchmark on UK Biobank. The benchmark contains 50,401 clinical questions from 4,739 participants (including 4,850 held-out questions from 455 participants), each linked to a patient environment with baseline and follow-up MRI acquired a median of 2.26 years apart across five organ regions and seven sub-modalities.

4. Learning beyond teacher imitation. Starting from action-level teacher trajectories, the compact policy is optimized on its own environment interactions without direct answer supervision, using outcome reward, privileged on-policy self-distillation, and rubric-based LLM feedback; controlled ablations isolate each component.

Main Findings

• CASE-8B leads the comparison panels. Averaged over the three roles, CASE (8B) reaches 76.1 Micro and 75.1 Macro accuracy. The untrained Qwen3-VL-8B base scores 46.0 Micro / 47.1 Macro.

• Reported relative gains over frontier models. Under matched evaluation conditions, the Qwen3-VL-8B-based agent achieves over 16% and 10% relative improvements in answer accuracy over GPT-5.4 (65.3 Micro / 65.7 Macro) and Claude Opus 4.8 (68.6 Micro / 65.8 Macro) respectively. Other compared systems include Kimi K3 (70.2/69.5), Qwen3.7-Plus (67.0/67.7), the Qwen3.8-Max teacher (59.7/58.3), Gemini-3.5-Flash (56.3/56.3), Kimi K2.6 (42.8/42.0), Lingshu-32B (49.9/50.3), Lingshu-7B (45.5/46.7), MedGemma-27B (27.8/27.5), and MedGemma-1.5-4B (40.5/41.4).

• Ablations show the components are complementary. From a shared RSFT initialization: RSFT alone 65.7 Micro / 65.0 Macro with a 0.43 insight score; + GRPO 69.3 / 68.2 / 0.47; + GRPO + RLAIF 70.6 / 69.5 / 0.63; + GRPO + OPSD 73.8 / 72.4 / 0.57; full CASE-8B 76.1 / 75.1 / 0.68, where the insight score is a [0,1] fixed external judge measure of the link between the Phase-4 insight and the final answer.

• OPSD prevents a late-training collapse. Without OPSD, GRPO+RLAIF restates earlier findings until the correctness reward collapses over the final third of the epoch (reported over steps 1,000–1,912 in Figure 3), whereas CASE-8B and GRPO+OPSD avoid this degeneration, and CASE-8B additionally converts the rubric signal into higher endpoint accuracy.

• Rubric feedback drives steady insight-quality gains. The black-box external-judge rubric produces a consistent rise in judged insight quality under three different judge models.

• Accuracy depends on acquired evidence, not priors. Varying what the agent may obtain (question-only, clinical-context-only, fixed-evidence) and removing each source in turn (MRI, follow-up, reports) lowers accuracy, separating evidence seeking from answer priors and from format or length artifacts.

• Efficiency is not bought with extra turns. For the tool-enabled Consultant, CASE finishes the longitudinal panels in few turns with a low answer-cutoff rate and a negligible invalid-JSON rate (reported in Table 17 alongside turns per episode and the fraction issuing at least one tool call).

• Demonstrated agent behavior. A rendered Consultant rollout shows the agent recalling a written skill, cross-checking segmentation, cross-visit comparison, and draft reports, then doubting an over-calling draft impression and requesting missing arterial and washout evidence before rejecting it; the Phase-4 insight and final answer agree and a post-answer review writes the conclusion back to skill memory.

Methodology in Plain English

The authors build a patient-level environment in which tools expose MRI views, Vista3D-derived measurements and overlays, and MedGemma-27B reports. Only two artifacts are precomputed once per participant (Vista3D whole-body Dixon MRI segmentation masks across six neck-to-ankle stations, and MedGemma-27B report objects); during reinforcement learning, mock tools replay observations keyed by an identical (participant, tool, argument) tuple. Tool selection, arguments, retrieval order, and stopping remain policy-generated.

Training has two stages. First, rejection-sampling fine-tuning (RSFT) distills screened teacher trajectories recorded with Qwen3.8-Max across three roles in LangGraph environments, retaining only trajectories with at least one tool call and a parseable Phase-4 insight whose offline judge score is at least 0.4, and training the full vision–language model end to end with next-token cross-entropy masked at non-assistant positions. Second, agentic reinforcement learning optimizes the policy on its own interactions using GRPO with G = 8 rollouts per prompt and no reference answers. One shared policy instantiates three roles — Radiologist, Anatomist, and Consultant — differing only in role prompt and tool registry, and all roles share a five-phase terminal response (plan, multi-source review, cross-visit comparison, key insight, answer derived strictly from it); this structures reasoning but does not prescribe tool order.

The reward combines an exact-match answer term (+1 exact, −1 wrong, −2 unparsed), a format term, a continuous rubric score from an external non-ancestor judge (Kimi K2.6), and a linkage term that rewards a highly rated insight only when the answer is also correct. Privileged on-policy self-distillation scores each rollout twice under the same frozen snapshot — once ordinarily and once with a hint supplying the reference answer, resolvable option text, and an optional recorded insight — and the resulting Causal-KL divergence (top-128 truncation) enters the return as a penalty before group normalization, so synthesis feedback shapes acquisition without tool-specific credit assignment.

Why This Matters

Impact on research. The work shifts medical agent evaluation from reasoning over preselected evidence to measurable evidence acquisition, and it shows that a compact 8B vision–language model can exceed much larger frontier systems when the harness, environment, and post-training recipe are matched. It also introduces a departure in distillation: the divergence between ordinary and answer-privileged views becomes a trajectory-level reward penalty entering the return before group normalization, rather than a differentiable loss or token/step/turn-level weight.

Real-world applications.

• Longitudinal radiology review, where an agent must decide which organ, sequence, visit, and matched slice to compare before reporting change.
• Clinical documentation and reporting assistants that gather and cross-check measurements, segmentation overlays, and prior reports rather than summarizing given text.
• Diagnostic triage support for patients with multiple follow-up imaging visits, where the determining evidence is spread across encounters.
• Auditing and benchmarking of clinical AI agents under matched evidence access and tool budgets.

Industry relevance. The benchmark and framework are positioned as a route from fixed-context assessment and workflow engineering toward learnable clinical investigation. Reproducibility plans include releasing implementation, recorded trajectories, benchmark construction and evaluation code, prompts, and training configurations, with the CASE Benchmark returned to UK Biobank and made available to approved researchers through the Returned Datasets Catalogue and Research Analysis Platform, and publicly releasable code hosted on Hugging Face and GitHub.

Future Directions

• Generalization beyond this cohort. The authors state that establishing utility beyond this cohort, including generalization and efficiency, remains future work.
• Broader toolsets and specialty-specific pathways. Real clinical practice spans wider toolsets and specialty-specific diagnostic pathways; the authors suggest the same evaluation and training paradigm can be instantiated by specifying the available evidence, tools, and decision contract.
• Unobservable labels and fallible components. Clinical labels may be unobservable from the available images, and generated evidence and judges remain fallible.
• Scoring precision. The privileged distillation scoring region targets synthesis rather than the answer, but its character-based span estimate does not guarantee exact Phase-4 isolation, as the authors note.

Target Audience

Researchers and engineers working on multimodal medical agents, agentic reinforcement learning, and benchmark design will benefit most, along with clinical AI practitioners interested in tool-use harnesses and longitudinal imaging evaluation. The paper is also relevant to groups studying on-policy distillation, privileged self-teaching, and rubric-based LLM feedback outside medicine, and to readers who need a concrete example of evaluating evidence acquisition rather than fixed-context question answering.

Authors’ abstract

Foundation models can serve as clinical agents through tool-use harnesses. However, conventional medical benchmarks assess reasoning over preselected evidence rather than the ability to seek it across clinical records and longitudinal imaging. We propose CASE: a series of role-specific Clinical Agents for Seeking Evidence, together with a tool-use harness and an agentic post-training framework for compact vision-language policy models. We further introduce a longitudinal multimodal benchmark built on UK Biobank, comprising 50,401 clinical questions derived from real-world ICD-10-coded diagnoses of 4,739 participants. Each question links to a patient-specific environment containing clinical context and multi-sequence MRI from baseline and follow-up visits, where agents autonomously select which visits, organs, modalities, slices, and specialist tools to inspect and compare. Supervised fine-tuning transfers evidence-seeking workflows from 14,734 frontier-model interaction trajectories, followed by agentic reinforcement learning on the learner's own environment interactions. Privileged on-policy self-distillation and rubric-based LLM feedback refine evidence-to-conclusion reasoning without prescribing tool sequences. Experiments show that CASE moves beyond question-answer imitation toward transferable investigation policies, strengthening evidence-grounded longitudinal reasoning. Under matched evaluation conditions, our Qwen3-VL-8B based agent achieves over 16% and 10% relative improvements in answer accuracy over GPT-5.4 and Claude Opus 4.8. Code will be available at https://github.com/VinyehShaw/CASE.