Dr. Assistant: Enhancing Clinical Diagnostic Inquiry via Structured Diagnostic Reasoning Data and Reinforcement Learning

AI summary

Overview

Research area: Clinical natural language processing — specifically, large language models for clinical decision support, diagnostic reasoning, and multi-turn clinical inquiry.

Technical level: Intermediate. The paper combines a structured clinical data format with a two-stage post-training pipeline (supervised fine-tuning followed by reinforcement learning). Readers need some familiarity with LLM post-training and reward-based optimization, but the clinical motivation and the dataset design are explained in accessible terms.

Scope: The paper introduces a structured representation of diagnostic reasoning logic (CDRD), a construction pipeline for it, a 14B-parameter clinical model (Dr. Assistant) trained on derived data, and a benchmark for evaluating diagnostic reasoning and inquiry in simulated clinical dialogue.

What This Paper Is About

Traditional Clinical Decision Support Systems rely on structured knowledge bases and rule-based algorithms, which are costly to maintain and adapt poorly to complex clinical situations. Although large language models perform well on static medical knowledge benchmarks, under zero-shot prompting they are weak at clinical inquiry, because effective inquiry requires professional diagnostic reasoning logic that is scattered across different chapters of clinical guidelines. The paper's goal is to capture that reasoning logic in a structured format, use it to train a model that both reasons and inquires in a clinically sound way, and build a benchmark that measures those abilities together rather than separately.

Key Contributions

1. Clinical Diagnostic Reasoning Data (CDRD): a triple-tuple representation, C = (S, E, D), capturing abstract diagnostic reasoning logic, where S is a core symptom, E is diagnostic evidence (associated symptoms, lab results, exam findings), and D is the list of differential diagnoses with their clinical manifestations and required examinations.

2. A three-stage CDRD construction pipeline that collaboratively combines LLM synthesis and physician refinement, proceeding through symptom extraction, disease matching, and logic completion, with outputs at every stage reviewed and refined by professional physicians.

3. Dr. Assistant, a clinical diagnostic model trained on Qwen3-14B, using a two-stage process: supervised fine-tuning on synthesized QA pairs, followed by reinforcement learning with a tailored reward function that scores clinical reasoning and inquiry skills while penalizing deviations from the source CDRD.

4. A clinical diagnostic reasoning and inquiry benchmark consisting of 242 real clinical cases and 147 real clinical inquiry rounds, designed to reflect the dynamic interaction between the inquiry process and the final diagnoses.

Main Findings

• ICD-Recall advantage at round 5: Dr. Assistant reaches 0.5066, a 6.70% relative improvement over the best closed-source result, Gemini-3-Pro-Preview (0.4748).

• Outperformance of open-source general models: Dr. Assistant exceeds Qwen3-32B (0.4356) by 16.30% relative, which the paper attributes to learning diagnostic reasoning logic and inquiry skills for inferring disease from a core symptom.

• Outperformance of open-source medical models: Dr. Assistant surpasses HuatuoGPT-o1-72B (0.4460) by 13.59% relative, reaching state-of-the-art ICD-Recall among open-source models. The paper attributes earlier medical models' weakness to a focus on domain knowledge without clinical diagnostic reasoning and inquiry skills.

• Precision is not the top result: Dr. Assistant's ICD-Precision is 0.4717, compared with 0.5211 for HuatuoGPT-o1-72B, 0.4842 for GPT-5, 0.5333 for Gemini-3-Pro-Preview, and 0.5159 for Grok-4-Fast. It averages 2.1983 candidate diagnoses per case, slightly higher than GPT-5's 1.7851. The authors describe this as high recall without candidate flooding, while noting that a missing correct candidate can cause missed diagnosis.

• Stable across inquiry turns: Across rounds 3, 5, and 10, Dr. Assistant has a mean ICD-Recall of 0.5060, standard deviation 0.0007, and coefficient of variation 0.14%, the lowest variance among all models listed.

• Higher physician satisfaction: In a double-blind evaluation by 11 physicians with 5 to 8 years of clinical experience (average around 6 years) and specialties including surgery, gynecology, internal medicine, and traditional Chinese medicine, Dr. Assistant improves on Med42-v2-8B by 35.83% and on HuatuoGPT-o1-72B by 26.78%, and advances GPT-5 by 15.19% (36.96 versus 21.77). Inter-rater agreement was a Cohen's Kappa of 0.955.

• General medical knowledge is retained: Dr. Assistant averages 0.7810 across CMMLU (eight medical-related subsets), MMLU-Pro, and MedQA, versus 0.7803 for Qwen3-14B — an overall retention rate of 100.10%. On the eight Chinese medical subsets of CMMLU, it matches or exceeds Qwen3-14B on five.

• Ablation results: Removing SFT drops satisfaction by 18.82% and ICD-Recall by 0.84% relative (0.5024); removing RL drops satisfaction by 6.80% and ICD-Recall by 29.00% relative (0.3927). Removing the fidelity penalty R_div drops satisfaction by 8.62% and ICD-Recall by 4.32% (0.4856); removing the clinical reasoning reward R_comp-r drops satisfaction by 5.50% and ICD-Recall by 3.54% (0.4893).

• Stronger on harder, multi-disease cases: On single-disease cases, Dr. Assistant scores 0.5426 versus 0.4139 for Baichuan-M2-32B (a 31.09% improvement) and versus GPT-5 (a 2.80% improvement). On multi-disease cases, it scores 0.5067 versus 0.3688 for Baichuan-M2-32B (37.39%) and versus 0.4652 for GPT-5 (8.92%).

• Training data scale: 11 CDRDs covering approximately 60 diseases yielded 1000 QA pairs each, totaling 11,000 examples for SFT, plus 36,688 turns of multi-turn inquiry dialogue for RL. All training data is in Chinese.

• Benchmark composition: The paper describes the benchmark as 242 real clinical cases and 147 inquiry rounds in the abstract and introduction, and as 242 patient profiles and 147 rounds of clinical inquiry dialogue records in the evaluation section; the department coverage is stated as 8 secondary departments, 9 sub-departments, and 7 sub-departments in different parts of the paper (including the figure caption), so the exact department count is reported inconsistently.

Methodology in Plain English

The researchers start from clinical guidelines, which contain the needed diagnostic logic but spread it across unrelated chapters. They convert each guideline into structured CDRD entries by running a three-stage pipeline: first an LLM proposes the symptoms in the guideline and physicians standardize them (for example, mapping "chest hurt" to "chest pain"); next the LLM proposes a list of diseases for each symptom, which physicians validate; finally the LLM extracts the supporting clinical evidence and completes the differential list for each symptom-disease pair. Physicians review every stage.

Because CDRDs are limited in number and static, the team uses them as seeds to generate training data. For supervised fine-tuning, they break each CDRD into semantically complete entries, rewrite each into a fluent answer, and then synthesize a matching question, producing QA pairs. For reinforcement learning, they set up a dual-agent simulation: a patient agent, given a real patient profile and the CDRD, plays a patient, and a physician agent produces a structured response each round with six reasoning steps (Known Information, User's Intention, Provided Information, Diagnoses, Information to be Collected, Response Strategy) plus a final inquiry.

The model is then fine-tuned on the QA pairs with standard negative log-likelihood loss, and afterward trained with reinforcement learning (DAPO) on the inquiry dialogues. The reward has two parts: a comparison reward that has a separate LLM-judge score each of the seven reasoning and inquiry components from 0 to 10 and combines them with weights (0.1, 0.3, and 0.6), and a fidelity penalty that subtracts points proportional to how many of the model's collected or produced information and diagnoses are absent from the source CDRD (weight 5). RL used Qwen3-32B as the judge, a learning rate of 1e-6, clipping parameters of 0.2 and 0.28, rollout number 4, batch size 512, and ran for 185 steps on 8 H800 GPUs with an additional 16 H800 GPUs for the judge. SFT used 4 epochs on 32 A800 GPUs. Data construction used DeepSeek-V3.2 for CDRDs and Qwen3-235B-A22B for QA pairs and dialogue data.

For evaluation, a model acts as a patient based on a real anonymized profile and interacts with the model under test. Diagnostic hypotheses produced at rounds 3, 5, and 10 are mapped to ICD-10 codes by an LLM and compared with the ground-truth diagnosis using a hierarchical similarity score, yielding ICD-Recall (the primary metric, measuring coverage of ground-truth diagnoses) and ICD-Precision. Separately, model inquiries generated from 147 real clinical inquiry records were reviewed by physicians, who judged relevance and expertise and declared a winner or a tie. All generation used greedy decoding with temperature 0.

Why This Matters

Impact on research. The paper argues that existing medical LLM evaluations — HealthBench and MAQuE among them — either focus on general consultation or separate the inquiry process from diagnostic reasoning, which does not reflect the way the two depend on each other in practice. By encoding diagnostic reasoning as a reusable structured object and tying reward directly to it, the work offers a template for injecting clinical logic into post-training rather than relying on knowledge coverage alone. It also shows that training on a vertical clinical task need not cause catastrophic forgetting of general medical knowledge, which is a common concern with post-training.

Real-world applications.

• Decision support for less experienced physicians who need guidance on what to ask and what diagnoses to consider during a consultation.
• Multi-turn clinical inquiry assistance, where the system proposes the next most informative question rather than only suggesting a final diagnosis.
• Triage or intake support, where differential diagnoses guide which information to collect before a physician sees the patient.
• Training and simulation for medical education, using the patient-simulator setup to practice diagnostic interview skills.

Industry relevance. The model is a 14B-parameter system trained on Qwen3-14B, which is small relative to many competing medical and general models, and the paper reports it outperforming far larger open-source models such as HuatuoGPT-o1-72B on the primary diagnostic reasoning metric. For organizations deploying clinical support tools, this suggests competitive clinical inquiry behavior may be achievable without the largest models, provided the reasoning structure and reward design are right. The paper's stated motivation — the high cost and limited adaptability of rule-based CDSSs — is a direct commercial pain point.

Future Directions

• Reduce reliance on manual physician effort. The authors note that CDRD construction depends on physician refinement, which is resource intensive and limits scalability; more automated or data-driven approaches are proposed as future work.
• Larger and broader evaluation. The paper states that the 242-case and 147-round test set is small and covers a limited number of departments, and calls for validation on larger datasets plus systematic analysis across different numbers of inquiry turns.
• Improve diagnostic precision and candidate prioritization. Dr. Assistant does not achieve the highest ICD-Precision and averages 2.1983 candidate diagnoses per case; the authors identify optimizing candidate prioritization as a direction.
• Better reward modeling and real clinical outcomes. The paper suggests that the current reward function may not capture all nuances of real diagnostic reasoning, and proposes explicit clinical outcome feedback or more automated reward modeling.
• Deployment conditions. Performance in fully open-ended clinical conversations and integration with live electronic health record systems remains untested.

Target Audience

Researchers and engineers working on medical LLMs, clinical decision support, and reinforcement learning from structured domain signals; clinical informatics teams evaluating deployable diagnostic assistance; and clinicians or medical educators interested in how diagnostic reasoning logic can be formalized and used to train systems that generate inquiries rather than only answers. Readers focused on benchmark design for multi-turn clinical dialogue or on preventing knowledge forgetting during vertical post-training will also find the setup relevant.

Authors’ abstract

Clinical Decision Support Systems (CDSSs) provide reasoning and inquiry guidance for physicians, yet they face notable challenges, including high maintenance costs and low generalization capability. Recently, Large Language Models (LLMs) have been widely adopted in healthcare due to their extensive knowledge reserves, retrieval, and communication capabilities. While LLMs show promise and excel at medical benchmarks, their diagnostic reasoning and inquiry skills are constrained. To mitigate this issue, we propose (1) Clinical Diagnostic Reasoning Data (CDRD) structure to capture abstract clinical reasoning logic, and a pipeline for its construction, and (2) the Dr. Assistant, a clinical diagnostic model equipped with clinical reasoning and inquiry skills. Its training involves a two-stage process: SFT, followed by RL with a tailored reward function. We also introduce a benchmark to evaluate both diagnostic reasoning and inquiry. Our experiments demonstrate that the Dr. Assistant outperforms open-source models and achieves competitive performance to closed-source models, providing an effective solution for clinical diagnostic inquiry guidance. Project information can be found at: https://github.com/YGswu/Dr.-Assistant .