Overview of CHIP 2025 Shared Task 2: Discharge Medication Recommendation for Metabolic Diseases Based on Chinese Electronic Health Records

AI summary

Overview

Research area: Clinical natural language processing (NLP) applied to Chinese electronic health records (EHRs), specifically automatic discharge medication recommendation for chronic metabolic diseases.

Technical level: Advanced. The task involves multi-label recommendation over heterogeneous clinical text, and the strongest reported systems were large language model (LLM)-based ensembles.

Scope: This paper is a competition overview describing the CHIP 2025 Shared Task 2, its CDrugRed dataset, participant turnout, and the final leaderboard results — it is not a single-method research paper.

What This Paper Is About

Discharge medication recommendation matters for patients with chronic metabolic diseases because it supports continuity of treatment after a hospital stay, helps prevent readmission, and contributes to long-term disease management. The CHIP 2025 Shared Task 2 challenged teams to automatically recommend appropriate discharge medications from real-world Chinese EHR data. This overview paper documents how the task was built, who took part, and how well the best systems performed.

Key Contributions

1. A new dataset: The organizers constructed CDrugRed, described as a high-quality dataset of 5,894 de-identified hospitalization records from 3,190 patients in China, intended for discharge medication recommendation research.
2. A defined shared task: They framed discharge medication recommendation for metabolic diseases as a competition problem using real-world Chinese EHRs, with Phase A and Phase B leaderboards.
3. A large-scale community evaluation: The task attracted 526 registered teams, with 167 submitting valid Phase A results and 95 submitting valid Phase B results.
4. A benchmark result and assessment: They report the top team's performance on the final test set and use it to characterize both the promise and the remaining difficulty of applying LLMs to this problem.

Main Findings

• Dataset scale: CDrugRed contains 5,894 de-identified hospitalization records drawn from 3,190 patients in China. The abstract does not describe the label set, record length, or train/validation/test splits.
• Task difficulty comes from three sources: medication recommendation is inherently multi-label (multiple drugs may be appropriate at once), clinical text is heterogeneous, and treatment plans vary from patient to patient.
• Strong participation: 526 teams registered; 167 submitted valid results to the Phase A leaderboard, and 95 submitted valid results to Phase B. The abstract does not explain how the two phases differ.
• Top performance: The best team achieved a Jaccard score of 0.5102 and an F1 score of 0.6267 on the final test set — the highest overall performance reported.
• Winning approach style: The abstract attributes the top result to advanced LLM-based ensemble systems, indicating that combining large language models was the most successful strategy observed.
• Overall takeaway: The organizers conclude that these results demonstrate both the potential and the unresolved challenges of applying LLMs to medication recommendation in Chinese EHRs.
• Ongoing evaluation: A post-evaluation phase remains open, with the competition page linked in the abstract.

Methodology in Plain English

The organizers started by assembling a dataset of real hospitalization records from China, removing identifying information, and pairing each discharge with the medications that were actually recommended. They then turned this into a competitive benchmark: teams would receive patient records and attempt to predict the correct discharge medications, and their predictions would be scored against the real prescriptions.

The task ran in two phases (Phase A and Phase B), each with its own leaderboard, so that teams could submit results and be ranked at more than one point in the competition. Participation was open — 526 teams registered, though only a fraction submitted valid entries at each phase.

The paper then compares how teams performed. The headline result comes from the team that ranked highest on the final test set. The abstract states that this top system used an ensemble of large language models, but it does not describe the architecture, prompting strategy, training data, or how the ensemble was combined. Details of individual team methods are not available in the abstract.

Why This Matters

Impact on research: The paper offers the clinical NLP community a shared, real-world Chinese-language benchmark for a clinically meaningful prediction task, plus a reference performance level (Jaccard 0.5102, F1 0.6267) that future work can be measured against. It also signals that LLM-based ensembling is currently the strongest known direction for this problem, while making clear that substantial headroom remains.

Real-world applications:

• Supporting clinicians at discharge by suggesting medications to consider for patients with chronic metabolic diseases.
• Reducing medication omissions or inconsistencies at the transition from hospital to home care.
• Helping standardize discharge planning across hospitals where documentation styles and prescribing conventions differ.
• Providing a basis for continuity-of-care tools that carry treatment plans forward between providers.

Industry relevance: Health IT vendors, hospital informatics teams, and companies building clinical decision support or EHR-integrated AI tools have a direct interest in this benchmark, since it targets the practical problem of turning messy, heterogeneous Chinese clinical text into actionable prescribing recommendations.

Future Directions

• Closing the performance gap: With F1 around 0.63, the top system still misses a large share of correct medications; better modeling of the multi-label structure and of patient-specific variability is an open problem.
• Explaining what worked: The abstract credits LLM-based ensembles but gives no methodological detail, leaving open questions about which ensemble designs, model choices, or prompting strategies drive performance.
• Robustness and generalization: It is not established in the abstract how these systems would perform across different hospitals, regions, specialties, or time periods, or how sensitive they are to the heterogeneous text the task highlights.
• Clinical validation and safety: Translating benchmark scores into practice requires understanding error types, whether misses or false suggestions are more harmful, and how such recommendations should be presented to clinicians.
• Continued evaluation: The post-evaluation phase remains open, which may allow further submissions and results beyond those reported here.

Target Audience

This paper is most useful to clinical NLP and medical informatics researchers, especially those working with Chinese-language EHR data or on medication and treatment recommendation. It also benefits shared-task organizers and benchmark designers who want a model for structuring a multi-phase clinical competition; data scientists at hospitals and health IT companies evaluating whether LLM-based approaches are ready for prescribing support; and machine learning practitioners entering the CHIP shared task series who need to understand the task setup, dataset, and the performance bar to beat.

Authors’ abstract

Discharge medication recommendation plays a critical role in ensuring treatment continuity, preventing readmission, and improving long-term management for patients with chronic metabolic diseases. This paper present an overview of the CHIP 2025 Shared Task 2 competition, which aimed to develop state-of-the-art approaches for automatically recommending appro-priate discharge medications using real-world Chinese EHR data. For this task, we constructed CDrugRed, a high-quality dataset consisting of 5,894 de-identified hospitalization records from 3,190 patients in China. This task is challenging due to multi-label nature of medication recommendation, het-erogeneous clinical text, and patient-specific variability in treatment plans. A total of 526 teams registered, with 167 and 95 teams submitting valid results to the Phase A and Phase B leaderboards, respectively. The top-performing team achieved the highest overall performance on the final test set, with a Jaccard score of 0.5102, F1 score of 0.6267, demonstrating the potential of advanced large language model (LLM)-based ensemble systems. These re-sults highlight both the promise and remaining challenges of applying LLMs to medication recommendation in Chinese EHRs. The post-evaluation phase remains open at https://tianchi.aliyun.com/competition/entrance/532411/.