Scalable Stewardship of an LLM-Assisted Clinical Benchmark with Physician Oversight

AI summary

Overview

Research area: Benchmark quality and evaluation methodology for large language models in clinical/medical domains, with a controlled reinforcement-learning experiment on label quality.

Technical level: Intermediate. The clinical scoring task itself is easy to grasp, but the paper uses symmetric mean absolute percentage error (sMAPE), bootstrap confidence intervals, supermajority voting over sampled LLM outputs, and Group Relative Policy Optimization (GRPO).

Scope: The paper audits label reliability in MedCalc-Bench, a widely used benchmark for computing medical scores from patient narratives, and proposes a three-phase physician-in-the-loop stewardship pipeline that re-labels the benchmark and measures how the original label errors distort both model evaluation and RL training.

What This Paper Is About

MedCalc-Bench, the most widely used public benchmark for computing medical scores such as LACE, CURB-65, and CHA₂DS₂-VASc from free-text clinical narratives, was built partly with LLM assistance: GPT-3.5 matched contexts to calculator questions, GPT-4 extracted clinical features, and custom Python scripts aggregated them. The paper asks whether those machine-derived reference labels are actually correct, and finds that a substantial fraction are not. It then builds a repeatable, scalable process for repairing such labels while spending as little physician time as possible, and measures the consequences of the errors on both how models are graded and how models are trained.

Key Contributions

1. A three-phase stewardship protocol for LLM-assisted benchmarks, organized around phased resource commitment: an automated audit, an independent blinded recomputation, and targeted physician adjudication reserved for the highest-disagreement cases.
2. A quantified audit of MedCalc-Bench v1.0 labels, finding that at least 27.3% of the 1,047 test instances are likely mislabeled, supported by two methodologically distinct phases that converge on similar rates (26.6% from the audit, 27.3% from recomputation).
3. Direct validation against physician ground truth on 50 contentious instances, showing recomputed labels agree with physicians 74% of the time (95% CI, 60–84%) versus 20% (95% CI, 11–33%) for the originals.
4. A controlled RL experiment isolating label choice as the only variable, demonstrating that training rewards derived from corrected labels produce a 13.5 percentage point gain (95% CI, 10.6–16.6%) on physician-labeled instances, with no degradation on related medical tasks.

Main Findings

• The original labels are substantially wrong. Among the 887 test instances with high-confidence recomputed labels, 220 (24.8%) differ from the original by more than the benchmark's own ±5% tolerance, and another 66 (7.4%) are flagged N/A. Combined, at least 27.3% of the 1,047 original test instances are likely mislabeled. The authors call this plausibly an underestimate, since the 160 instances where agents could not reach a supermajority are deferred as hard cases.
• Two independent phases agree on the error rate. Phase 1's audit flagged 279 of 1,047 instances (26.6%) as Likely Error, touching 40 of the 55 distinct scoring questions in the test set. Phase 2, which sees no original metadata, produced a similar estimate.
• Three recurring failure modes. Feature extraction errors (for example, confusing hemoglobin with albumin and extracting a physiologically impossible value), incorrect aggregation logic (for example, a Glasgow Coma Scale script double-counting a feature), and instances where the question is not answerable from the given context (for example, a sodium correction for hyperglycemia applied to a hypoglycemic patient).
• Physician spot-check validated the auditor. A physician author reviewed 7 flagged instances in his clinical domain and agreed with the error identification in 7/7 reviews.
• Recomputed labels match physicians far better. On the 50-instance physician-adjudicated subset, recomputed labels agreed 37 out of 50 times (74%) versus 10 out of 50 (20%) for originals. sMAPE dropped from 72.7% (95% CI, 47.9–99.1) to 20.1% (95% CI, 8.0–35.9).
• Original labels understate frontier model capability. Grading four frontier models in February 2026 against the original labels on the same 887 instances yields accuracies of 65.7%, 69.4%, 62.5%, and 67.8% — a drop of 16–23 percentage points relative to grading against the recomputed labels. The paper suggests Anthropic's "Claude for Healthcare" launch figures were likely obtained using the original v1.0 labels.
• Label quality changes training outcomes. Two identical copies of Qwen3-8B were trained with GRPO on 4,593 training instances, differing only in whether rewards came from original or recomputed labels. Both began at identical 28% accuracy. During the final training phase (steps 205–300), the model trained on recomputed labels reached 51.9% mean accuracy (95% CI, 50.5–53.3) versus 38.4% (95% CI, 36.7–39.9) — a gap of about 13.5 percentage points.
• The training gap narrows on the full test set but persists elsewhere. On the 887-instance held-out set graded against recomputed labels, the gap was +8.7 points (71.4% vs 62.6%). On three additional medical evaluation sets (MedQA and two recent MedCalc-Bench revisions), the advantage of better labels persisted at roughly 1 percentage point (p = 0.001), and did not degrade performance on any related task.

Methodology in Plain English

The organizing principle is that GPU hours scale but physician hours do not, so the pipeline spends expert attention only where it is most likely to change an answer.

Phase 1 — Automated audit. An LLM auditor (Gemini 2.5 Pro, June 2025 version) with web-search access reviewed each of the 1,047 test labels along with the metadata describing how the original label was derived. It checked the scoring rule, verified that the structured feature values matched the narrative, and ran sanity calculations. Each instance was audited five times independently at temperature 1, and an instance was flagged only if at least 4 of 5 runs called it clinically inconsistent. A physician then spot-checked 7 flagged cases.

Phase 2 — Independent recomputation. A separate agent, given only the patient context and the question and blinded to the original label and its metadata, recomputed every score from scratch using web search for scoring rules and a Python sandbox for arithmetic. It was instructed to answer "N/A" rather than guess when inputs were missing or the question did not fit the context. Five independent copies ran per instance, with a supermajority rule for high-confidence labels plus a near-consensus allowance (3 of 5, provided one remaining output was within ±5% for numerics or ±1 day for dates). This ran over the full test split and a stratified random sample of the training split (5,183 instances, roughly 50% of 10,053, stratified by calculator question type).

Phase 3 — Physician adjudication. Test instances were ranked by disagreement between original and recomputed labels, treating any abstention as maximal disagreement. Fifty were sampled and assigned to physician authors by domain fit, each reviewed once, single-blind, by one physician who recomputed the score from first principles and could abstain.

Controlled RL experiment. Two identical Qwen3-8B checkpoints were trained with GRPO on the same 4,593 instances, prompts, hyperparameters, and compute budget, varying only whether the reward came from original or recomputed labels. Evaluation used the 50 physician-labeled instances as the primary measure, the 887-instance held-out set graded against recomputed labels, and three additional medical evaluation sets.

Why This Matters

The paper reframes benchmarks as living documents rather than static artifacts, and shows that label noise is not benign measurement error — it is directional distortion that changes conclusions about model capability and changes what models learn when benchmark labels are reused as reward signals. The finding that original labels understate frontier model accuracy by 16–23 percentage points is directly relevant to anyone reading published medical benchmark numbers, including industry launch materials.

Real-world applications:

• Clinical AI procurement and evaluation. Hospitals and health systems comparing medical LLMs can account for the fact that a widely cited benchmark may understate real model performance.
• Diagnostic and triage support tools. The paper argues a safely useful system should flag out-of-distribution cases for human review and abstain rather than hallucinate an answer; improving label quality in effect teaches models when to say "N/A."
• Post-training and alignment pipelines. Anyone using benchmark labels as RL rewards inherits the benchmark's errors as an implicit objective, as the 13.5 percentage point training gap demonstrates.
• Regulatory and safety documentation. The protocol offers a template for documenting label provenance, assumptions, and revisions in safety-critical evaluation sets.
• Other expert-label domains. The authors argue the design principle transfers to legal reasoning, materials science, and financial risk assessment, wherever expert labels are expensive and benchmark accuracy is safety-relevant.

Industry relevance: MedCalc-Bench has been adopted broadly in academic research and was one of two medical benchmarks evaluated by Anthropic in its "Claude for Healthcare" launch, which the paper's reproduction analysis suggests used the original v1.0 labels. The benchmark is reported at 11,100 labeled instances across 55 calculators in the abstract, with the methods section describing 11,000 context-question pairs (10,053 training; 1,047 test) covering 55 score questions.

Future Directions

• Reconciling dataset versions. The authors audited v1.0 for comparability, and note that MedCalc-Bench v1.1 and v1.2 began releasing during final manuscript preparation, after the auditing and RL experiments were completed. A systematic analysis of performance discrepancies between versions is pointed to in the supplementary appendices but not resolved here.
• Recycling unanswerable instances. Rather than discarding instances where the question cannot be resolved from the context, the paper proposes keeping the context-question pair and revising the ground-truth label to N/A, which would require the benchmark's inference prompt to acknowledge that abstention is sometimes the correct answer.
• Handling the 160 deferred hard cases. These instances never reached supermajority agreement and are excluded from the high-confidence set; the paper notes the true mislabel rate could be substantially higher once they are accounted for, leaving open how to label them.
• Generalizing and scaling the protocol. The paper's claims rest on a 50-instance physician-adjudicated subset, which the authors describe as modest. Extending physician adjudication and confirming whether the stewardship design transfers to other expert-label domains remains open.

Target Audience

This paper is most useful to benchmark builders and maintainers, AI evaluation researchers, and clinical AI developers who rely on published medical benchmarks for model selection or post-training. It also speaks to clinicians involved in ground-truth labeling, since it is explicitly designed around minimizing their time commitment, and to regulators and industry teams who need to reason about how much confidence a benchmark score deserves.

Authors’ abstract

Reference labels for machine-learning benchmarks are increasingly synthesized with LLM assistance, but their reliability remains underexamined. We audit MedCalc-Bench, a clinical benchmark for medical score computation whose labels were partly derived with LLM assistance, and develop a scalable physician-in-the-loop stewardship pipeline to reassess them. At least 27% of test labels are likely erroneous or incomputable. On a 50-instance subset validated by physicians, our recomputed labels agree with physician ground truth 74% of the time (95% CI, 60-84%) versus 20% for the originals (95% CI, 11-33%). Using original labels to evaluate frontier LLMs underestimates accuracy by 16-23 percentage points. In a controlled reinforcement-learning experiment, a model trained on recomputed labels outperforms one trained on originals by 13.5 percentage points (95% CI, 10.6-16.6%) on physician-labeled instances, and this advantage extends to related medical tasks. LLM-assisted benchmarks can propagate systematic errors into both evaluation and post-training unless actively stewarded.