CURE-Med: Curriculum-Informed Reinforcement Learning for Multilingual Medical Reasoning

AI summary

Overview

Research area: Multilingual medical reasoning in large language models (LLMs), combining supervised fine-tuning and reinforcement learning (RL) with multilingual benchmark construction.

Technical level: Advanced. The paper assumes familiarity with supervised fine-tuning, chain-of-thought reasoning, Group Relative Policy Optimization (GRPO), reward modeling, and LLM-as-a-judge evaluation.

Scope: The paper introduces CureMed-Bench, a 15,774-instance, 13-language open-ended medical reasoning benchmark with single verifiable answers, and Cure-Med, a two-stage code-switching-aware SFT plus curriculum-informed GRPO framework evaluated against 28 baseline models.

What This Paper Is About

LLMs perform well on monolingual mathematical and commonsense reasoning but remain unreliable for multilingual medical reasoning, which hinders deployment in multilingual healthcare settings. The paper addresses two recurring failure modes: reduced logical accuracy and unstable language behavior (language drift) in languages other than English.

The goal is to build both a source of clinically grounded multilingual evaluation data and a training recipe that improves logical correctness and target-language fidelity at the same time, rather than trading one for the other.

Key Contributions

1. A systematic evaluation of multilingual medical reasoning using open-ended medical queries with verifiable answers, enabling independent measurement of logical accuracy and language consistency across languages.

2. CureMed-Bench, described as a large-scale multilingual medical reasoning dataset spanning 13 languages across high-, mid-, and low-resource settings, including underrepresented languages such as Amharic, Yoruba, and Swahili.

3. Cure-Med, a two-stage training framework that combines code-switching-aware supervised fine-tuning with curriculum-informed reinforcement learning to jointly optimize logical correctness and linguistic fidelity.

4. Automatic and human evaluation results showing state-of-the-art performance on CureMed-Bench, plus improved out-of-distribution generalization, including robustness in low-resource languages and stronger performance on unseen medical questions and languages.

Main Findings

• Headline results at 7B and 32B: Cure-Med achieves 85.21% language consistency and 54.35% logical correctness at 7B parameters, and 94.96% language consistency and 70.04% logical correctness at 32B parameters.

• Gains across every parameter scale: Cure-Med-Qwen2.5-1.5B reaches 57.60% consistency and 28.32% accuracy; Cure-Med-Qwen2.5-3B reaches 74.28% consistency and 42.93% accuracy; Cure-Med-Qwen2.5-14B reaches 90.27% consistency and 63.74% accuracy.

• Comparison against the strongest mid-scale baseline: At 7–9B, Cure-Med improves logical correctness to 54.35% versus 46.86% for the best baseline, HuatuoGPT-o1-8B, while maintaining 85.21% consistency.

• Small models beat much larger ones: Cure-Med-1.5B outperforms several baselines ranging from 7B to 70B, and Cure-Med-32B outperforms all 28 baseline models.

• Favorable language/reasoning trade-off: Baselines show a weak trade-off between language consistency and logical correctness; Cure-Med shifts performance toward the upper-right of the trade-off plot, indicating joint improvement rather than sacrificing one for the other.

• Smooth scaling behavior: Cure-Med scales language consistency from 57.6% at 1.5B to 95.0% at 32B, and logical correctness from 28.3% to 70.0%, whereas instruction-tuned baselines show only modest language-consistency gains as scale increases.

• Code-switched SFT vs. naïve SFT: Naïve SFT is small and sometimes unstable — at 3B, language consistency rises from 8.39% to 13.07% while logic accuracy falls from 10.83% to 9.50%. Code-switched SFT in Cure-Med raises 1.5B language consistency from 3.84% to 53.67% and logic accuracy from 6.20% to 22.97%, reaching 90.29% language consistency and 66.34% logic accuracy at 32B (without RL).

• Curriculum RL adds value over SFT and over naïve RL: Naïve RFT yields limited and uneven gains, especially at smaller scales. Cure-Med with curriculum RL delivers 28.32% accuracy at 1.5B versus 8.80% for naïve RFT, and 74.28% versus 13.28% consistency at 3B, improving across all model sizes relative to the without-RL variant (for example, 94.96% vs. 90.29% consistency at 32B).

• Out-of-distribution transfer: On MMedBench, the 1.5B backbone rises from 6.00 to 24.00 on French and from 20.00 to 57.50 on Russian. On MedExpQA, the 1.5B model rises from 1.40 to 44.80. On MedQA, the 1.5B model improves from 21.00 to 59.50 on Chinese variants. Gains persist at larger scales.

• Cross-benchmark standing: Cure-Med-32B achieves the best performance on CureMed-Bench (70.04%) and MMed-Bench (79.57%), and remains competitive on MedQA and MedExpQA, where HuatuoGPT-70B leads narrowly.

• Human validation: Across 13 languages, user studies report an average rating of 4.89/5, with all samples reviewed by native speakers and medical experts (physicians, advanced medical students, and nursing PhD candidates) for clinical correctness, linguistic fidelity, and cultural appropriateness.

• Benchmark positioning: CureMed-Bench is the only dataset in the paper's comparison table reported as simultaneously open-ended, with reasoning supervision, covering low-resource languages, spanning 13 languages, and containing 15,774 instances.

Methodology in Plain English

Building the data. The team started from MedlinePlus, a clinically validated medical resource curated by U.S. federal health agencies. They used GPT-4o to retrieve content and draft closed-ended multiple-choice questions in each of 13 target languages, with four options and exactly one correct answer. To keep only questions needing real reasoning, they discarded items answered correctly by all three compact models: Qwen2.5-3B, Qwen2.5-7B, and LLaMA-3.1-8B. GPT-4o was then used to flag questions with multiple valid answers or inconsistent cross-lingual interpretation. Remaining items were converted into open-ended prompts with an explicit reasoning chain and a free-form ground-truth answer, yielding a dataset of (question, reasoning, answer) triples with a single clinically grounded solution. Native speakers and medical experts then reviewed everything under IRB-approved procedures.

Stage one training. The model is first fine-tuned on code-switched long chain-of-thought trajectories, meaning intermediate reasoning steps may switch between languages while the final answer stays in the query's target language. This cold-start stage stabilizes multi-step reasoning before stricter language constraints are applied.

Reward design. Three rewards are combined with weights: a correctness reward from a GPT-4.1 verifier that returns a continuous score from 0 to 1, allowing partial credit for paraphrased but clinically valid reasoning; a binary language consistency reward for whether the whole output matches the query language; and a binary format reward that checks use of the required structure (<thinking>, numbered <step n>, and <answer> tags). A closed-source multilingual reward model that performs competitively on RewardBench is used, and a separate model performs LLM-as-a-judge verification to mitigate same-model judge bias.

Stage two training. After SFT, the model is trained with GRPO, using the standard optimization rule but organized into curriculum phases. The curriculum is organized by language resource availability rather than problem complexity, on the observation that high-resource languages give more stable reward signals early in RL. Three tiers are used: high-resource (French, Japanese, Spanish, Vietnamese), medium-resource (Korean, Thai, Turkish, Bengali), and low-resource (Amharic, Yoruba, Hausa, Hindi, Swahili). Training starts on the high-resource tier and expands downward when rewards plateau, retaining 85% (α = 0.85) of samples from the previous phase to reduce catastrophic forgetting.

Experimental setup. Backbones are Qwen-2.5-Instruct models at 1.5B, 3B, 7B, 14B, and 32B parameters. Training used eight NVIDIA A100 GPUs: three epochs of SFT on the code-switched dataset, then language-resource-aware curriculum fine-tuning with GRPO. Data was split 80% train / 20% held-out test, with the train set further divided into 80% for SFT and 20% for reinforcement fine-tuning. The 28 baselines span general-purpose models (Qwen2.5-Instruct, LLaMA, Gemma, Mistral, Apollo2, Ministral) and medical-specific models (MedAlpaca, Meditron, UltraMedical, HuatuoGPT, OpenBioLLM, BioMistral, MMed-LLaMA), all evaluated zero-shot across three independent runs. Evaluation used an LLM-as-a-judge framework with GPT-4o, measuring logical accuracy (clinical accuracy of the final answer) and language consistency (whether the final answer is in the question's target language).

Why This Matters

Impact on research. The work argues that language fidelity and reasoning correctness can be optimized jointly rather than traded off, and that curriculum design by language resource level changes scaling behavior for multilingual reasoning. It contributes an open benchmark and a codebase (cure_med) and shows that small models trained with this recipe can outperform far larger ones on this task.

Real-world applications:

• Multilingual clinical education and training tools where explanations must be readable in the learner's own language while medical terminology remains precise.
• Patient-facing health information systems serving speakers of Amharic, Yoruba, Hausa, Swahili, Hindi, Bengali, and other mid- and low-resource languages.
• Clinical decision support that must produce inspectable reasoning, since clinicians and patients need to understand how a conclusion was reached.
• Cross-lingual triage and initial screening in multilingual health systems, where a model that drifts into English mid-answer is unreliable for the user.

Industry relevance. The results matter to health systems, hospital IT, and LLM developers who need measurable language consistency alongside accuracy before deployment. The paper also discloses a practical constraint for industry: parts of the data generation and verification pipeline rely on API-based models, which the authors note can be costly and may hinder reproducibility for some researchers.

Future Directions

• Reduce dependence on proprietary APIs. Replacing generation and verification components with smaller open-source models trained for the same roles, and releasing prompts, code, and verifier alternatives to lower cost and improve reproducibility.
• Broaden coverage. Expanding language coverage and broadening clinical settings and modalities beyond the current single-answer, open-ended question format.
• Move beyond single-answer questions. Extending to longitudinal care trajectories, multi-visit decision-making, and multimodal clinical evidence, which the benchmark explicitly does not capture.
• Address uneven difficulty. Mitigating the uneven difficulty between high- and low-resource settings that arises from uneven availability of clinically reliable source material across languages.

Target Audience

Researchers working on multilingual NLP, medical NLP, and reinforcement learning for LLM reasoning; clinician informaticists and medical AI evaluators who need benchmarks with verifiable answers and human validation; practitioners and product teams deploying LLMs in multilingual healthcare contexts; and graduate students who want a worked example of combining supervised fine-tuning with curriculum-based GRPO under a multilingual reward design.

Authors’ abstract

While large language models (LLMs) have shown to perform well on monolingual mathematical and commonsense reasoning, they remain unreliable for multilingual medical reasoning applications, hindering their deployment in multilingual healthcare settings. We address this by first introducing CUREMED-BENCH, a high-quality multilingual medical reasoning dataset with open-ended reasoning queries with a single verifiable answer, spanning thirteen languages, including underrepresented languages such as Amharic, Yoruba, and Swahili. Building on this dataset, we propose CURE-MED, a curriculum-informed reinforcement learning framework that integrates code-switching-aware supervised fine-tuning and Group Relative Policy Optimization to jointly improve logical correctness and language stability. Across thirteen languages, our approach consistently outperforms strong baselines and scales effectively, achieving 85.21% language consistency and 54.35% logical correctness at 7B parameters, and 94.96% language consistency and 70.04% logical correctness at 32B parameters. These results support reliable and equitable multilingual medical reasoning in LLMs. The code and dataset are available at https://cure-med.github.io/