MedEyes: Learning Dynamic Visual Focus for Medical Progressive Diagnosis

AI summary

Overview

• Research area: Medical vision-language models (VLMs) and vision-grounded chain-of-thought reasoning, using reinforcement learning with verifiable rewards (RLVR) for medical visual question answering.
• Technical level: Advanced. The paper assumes familiarity with reinforcement learning (policy optimization, on-policy vs. off-policy learning, advantage normalization, entropy collapse) and with multimodal model training.
• Scope: The paper proposes MedEyes, a hybrid on-policy/off-policy reinforcement learning framework that trains a VLM to imitate clinician-style progressive visual focus — scanning for abnormal regions, then drilling into them — for medical diagnosis, and evaluates it on five medical VQA benchmarks.

What This Paper Is About

Medical diagnosis requires clinicians to repeatedly look at an image, form a hypothesis, focus on a specific region, and refine their impression. Existing vision-language models either memorize generic reasoning patterns through supervised fine-tuning, or, when trained with purely on-policy reinforcement learning, reinforce reasoning paths that sound coherent but are clinically wrong.

MedEyes attacks this by combining self-driven exploration with structured "expert" trajectories that act as cognitive anchors, so the model learns both to imitate expert visual search and to discover its own diagnostic strategies. The goal is a model that grounds every reasoning step in an actual image region and improves diagnostic accuracy across radiology, pathology, and multimodal medical questions.

Key Contributions

1. MedEyes, a dynamic-focus multi-round reasoning RL framework. It breaks with traditional medical post-training by introducing structured off-policy expert trajectories that guide the policy while avoiding the policy collapse seen in pure on-policy RL.
2. A collaboration between the Gaze-guided Reasoning Navigator (GRN) and the Confidence Value Sampler (CVS). The GRN reproduces diagnostic workflow through a scanning-and-drilling dual-mode strategy; the CVS uses nucleus sampling and adaptive termination to build a diverse, high-quality off-policy trajectory library.
3. A dual-stream GRPO optimization architecture. It isolates on-policy and off-policy learning signals, addressing reward assimilation and entropy collapse, and balances learning expert diagnostic patterns against task adaptability.
4. Comprehensive validation on five medical VQA benchmarks, reporting not only accuracy gains over existing methods but also gains in clinical interpretability and visual localization accuracy.

Main Findings

• Average accuracy of 65.9 across five benchmarks. MedEyes reaches 70.7 on VQA-RAD, 79.1 on SLAKE, 64.8 on PathVQA, 55.3 on PMC-VQA, and 59.7 on MMMU* (the Health & Medicine subset).
• It outperforms the strongest medical-specific baseline by 8.5%. The best medical-specific model compared is GMAI-VL at an average of 57.4 (64.6 VQA-RAD, 71.9 SLAKE, 47.2 PathVQA, 52.3 PMC-VQA, 51.2 MMMU*). The abstract frames this as an average improvement of +8.5pp.
• It outperforms the strongest reinforcement-learning baseline by 13.4%. MedVLM-R1 averages 52.5 (61.4 VQA-RAD, 65.9 SLAKE, 55.2 PathVQA, 44.8 PMC-VQA, 35.5 MMMU*).
• General VLMs trail substantially. GPT-4o averages 51.1, InternVL-2 51.7, and Qwen2.5-VL-3B 46.1. Med-R1 averages 48.5, GRIT† 49.3, DeepEyes† 50.5, Med-Flamingo 47.7, LLaVA-Med 43.7, MedVInT 39.0, and RadFM 35.4. Gemini-Pro is listed as a baseline but no benchmark numbers for it appear in the results table.
• Removing off-policy learning causes the largest ablation drop. The full model averages 71.5 on the three-benchmark ablation set; removing off-policy trajectories drops it to 61.0 (a reported 10.5% degradation).
• GRN removal causes an 8.7% drop. Without GRN the average falls to 62.8, confirming that the scanning–drilling strategy is what replicates expert workflow.
• CVS removal causes a 5.5% drop. Without CVS the average falls to 66.0, indicating that credible diverse trajectory sampling matters for performance.
• Neither exploration mode alone is sufficient. Scanning-only averages 66.6 and drilling-only averages 65.6, both below the full 71.5. Scanning-only is reported to fail on fine-grained tasks; drilling-only lacks systematic exploration.
• Six expert trajectories is the sweet spot. Averages rise from 64.4 (2 trajectories) to 67.0 (4) to 71.5 (6), then fall slightly to 71.1 (8).
• A three-step reasoning sequence performs best in the ablation. Averages are 71.2 (2 steps), 71.5 (3 steps), 70.9 (4 steps), and 69.0 (5 steps); longer sequences are reported to accumulate errors. Note that the implementation details separately state a maximum length of T_max = 4 for CVS termination.
• The reward weighting matters. Accuracy-only (1.0/0.0/0.0) averages 68.9, while the selected 0.7/0.2/0.1 split of accuracy/grammar/diversity rewards averages 71.5; other splits range from 69.5 to 70.8.
• Trajectory quality beats trajectory quantity. Using GRN+CVS trajectories averages 71.5 versus 58.7 for random sampling, 63.9 for DeepSeek-R1-generated trajectories, 63.5 for recency-based historical replay, and 65.6 for reward-oriented historical replay.
• Training shows an exploration-to-efficiency transition. The reward curve rises steadily with the largest improvement between steps 200 and 800, then stabilizes. Average trajectory length first grows from 2.1 steps to roughly 3.0 steps, then settles at 2.6 steps.
• Qualitative case study. In a T2-weighted MRI liver case, MedEyes first localizes bilateral kidneys as anatomical landmarks, then drills into the region adjacent to the right kidney; cross-attention heatmaps show attention shifting from diffuse activation to diagnostically salient areas.
• Two failure modes are documented. Quantitative measurement tasks suffer from flawed relative scaling, substituting anatomical proportion estimation for pixel-to-centimeter calibration; and fine-grained concept differentiation fails, with the model occasionally conflating pathological subtypes such as "aneurysmal changes" versus "dissecting aneurysms."

Methodology in Plain English

MedEyes is trained in two connected streams. The on-policy stream is the model itself, generating its own multi-round diagnostic traces. The off-policy stream is a set of synthetic "expert" traces produced by a separate procedure and used as a reference for how a clinician would search.

That expert procedure has two parts. The Gaze-guided Reasoning Navigator (GRN) mimics eye-tracking-derived clinical search behavior with two modes: a scanning mode that asks a visual expert model to locate all abnormal regions in the image and assign confidence scores, and a drilling mode that asks for a detailed analysis of one candidate region. If drilling raises a region's confidence by at least a threshold (δ = 0.15, with a stability constant ε = 1×10⁻⁶), the model keeps drilling; otherwise it returns to scanning. Each step is formatted with standard tags — <reasoning>, <action> with a "Gaze" coordinate, <feedback> with a cropped region, and finally <answer> — so traces are structurally comparable.

The Confidence Value Sampler (CVS) then turns each GRN run into several alternative exploration paths using nucleus sampling (p₀ = 0.9), stopping when regional confidence exceeds ξ = 0.85 or the sequence hits a maximum length (T_max = 4). This yields variable-length trajectories that reflect that some cases are obvious and others require extensive multi-region search. Six such trajectories (N_expert = 6) form the off-policy replay buffer.

Training uses GRPO, a reinforcement learning algorithm, with a composite verifiable reward: an accuracy reward (does the final answer match ground truth), a grammar reward (is the multi-round tag structure well-formed, binary 1 or 0), and a diversity reward (are multiple distinct, spatially separated regions explored, measured with IoU). The distinctive twist is that advantage normalization is computed separately for on-policy and off-policy trajectories rather than pooled, so that expert traces cannot dominate or drown out the model's own learning signal.

The base model is Qwen2.5-VL-3B, with MedPLIB used as the sole visual expert, n = 5 candidate regions per step, images processed at 336×336 with patch size 14, reward weights λ_acc = 0.7, λ_grammar = 0.2, λ_div = 0.1, AdamW optimization at a learning rate of 1×10⁻⁶, three epochs on six NVIDIA GeForce RTX 3090 GPUs, 80 iterations, a rollout batch size of 98, and 8 rollouts per prompt.

Why This Matters

Impact on research. The paper argues that purely on-policy RLVR in medical settings reinforces superficially coherent but clinically inaccurate reasoning, and shows that structured off-policy expert guidance can train weak initial models that pure on-policy RL cannot handle. The dual-stream advantage decoupling is a general recipe that could transfer to other domains where expert trajectories exist but behavioral cloning alone is insufficient. The work also ties reasoning steps explicitly to image regions, making the reasoning trace auditable rather than purely textual.

Real-world applications (as supported or implied by the paper):

• Clinical decision support for radiology, where a model that points at the region driving its answer is more checkable than one that only emits text.
• Pathology slide review, since PathVQA is among the evaluated benchmarks and the model is measured on histological imagery.
• Automated medical VQA triage and report assistance across CT, MRI, X-ray, and multimodal medical questions.
• Training and auditing tools that expose progressive visual search patterns, since the cross-attention visualizations map reasoning tokens to image regions.

Industry relevance. The framework is built on a 3B-parameter open model and trained on six consumer-grade RTX 3090 GPUs, which is a comparatively modest compute footprint. That makes the approach plausible for organizations that cannot train frontier medical models from scratch. The explicit failure analysis around quantitative measurement is also directly relevant to regulated clinical use, where pixel-to-centimeter calibration errors and subtype confusion have real consequences.

Future Directions

• Richer tooling for quantitative measurement. The authors state that flawed relative scaling — substituting anatomical proportion estimation for pixel-to-centimeter calibration — introduces variance incompatible with clinical requirements, and that richer tools are needed.
• Deeper expertise understanding for fine-grained concept differentiation. The model conflates pathological subtypes with shared morphological features (e.g., "aneurysmal changes" vs. "dissecting aneurysms"), which the authors identify as an open challenge for future medical agents.
• Reconciling the trajectory-length findings. The ablation reports 3 steps as optimal while the implementation specifies a maximum of 4 steps, and training dynamics show length falling from ~3.0 to 2.6; determining the right adaptive stopping policy is an open question.
• Extending beyond the five evaluated benchmarks. The paper does not report training dataset statistics in the main text (they are deferred to supplementary materials), and generalization to broader imaging modalities and clinical workflows remains to be demonstrated.

Target Audience

Researchers and engineers working on medical vision-language models, multimodal reasoning, and reinforcement learning for LLM/VLM post-training will get the most from this paper — particularly those interested in off-policy data integration, GRPO variants, and reward design. Clinician-scientists involved in validating AI diagnostic tools will find the failure-case analysis and visual grounding discussion relevant. Practitioners without a background in policy optimization will find the methodology dense; the results tables and case study are the accessible entry points.

Authors’ abstract

Accurate medical diagnosis often involves progressive visual focusing and iterative reasoning, characteristics commonly observed in clinical workflows. While recent vision-language models demonstrate promising chain-of-thought (CoT) reasoning capabilities via reinforcement learning with verifiable rewards (RLVR), their purely on-policy learning paradigm tends to reinforce superficially coherent but clinically inaccurate reasoning paths. We propose MedEyes, a novel reinforcement learning framework that dynamically models clinician-style diagnostic reasoning by progressively attending to and interpreting relevant medical image regions. By incorporating off-policy expert guidance, MedEyes converts expert visual search trajectories into structured external behavioral signals, guiding the model toward clinically aligned visual reasoning. We design the Gaze-guided Reasoning Navigator (GRN) to emulate the diagnostic process through a dual-mode exploration strategy, scanning for systematic abnormality localization and drilling for detailed regional analysis. To balance expert imitation and autonomous discovery, we introduce the Confidence Value Sampler (CVS), which employs nucleus sampling and adaptive termination to create diverse yet credible exploration paths. Finally, the dual-stream GRPO optimization framework decouples on-policy and off-policy learning signals, mitigating reward assimilation and entropy collapse. Experiments demonstrate that MedEyes achieves an average performance improvement of +8.5pp across multiple medical VQA benchmarks, validating MedEyes's potential in building trustworthy medical AI systems. Code is available at https://github.com/zhcz328/MedEyes.