The Pulse
Clinical AI Review Finds Doctors Missed 26.6% of Erroneous ECG Readings
A systematic review in npj Digital Medicine found that clinicians failed to correct or recognize 26.6% of erroneous interpretations across four real-world computerized ECG studies. The authors say human oversight should be measured by wheth
AI.info Team · Written with AI · fact-checked ·
Clinicians failed to correct or recognize 26.6% of erroneous computer-generated ECG interpretations in four real-world studies, according to a systematic review published October 23 in npj Digital Medicine. The estimate comes with a wide confidence interval—6.4% to 65.6%—and substantial variation among the studies, a reminder that human review is not a uniform safety net.
The review examined how clinicians respond when automated systems give incorrect advice, drawing on 46 reports representing 41 independent cohort clusters. Its authors, Di Zeng, Jianyong Lei and Zhihui Li, conclude that oversight needs to be tested by whether it catches errors, rather than assumed to make a system safe simply because a clinician is involved.
The ECG estimate comes with wide uncertainty
The 26.6% figure applies specifically to erroneous interpretations in four real-world computerized ECG studies. Their results varied sharply: the pooled estimate’s 95% confidence interval ran from 6.4% to 65.6%, and the statistical measure of between-study heterogeneity was high (I² = 95.8%). That spread makes the pooled number a warning signal, not a reliable forecast for any particular hospital or ECG tool.
Across three adjusted cohorts, the odds of a clinician reaching a correct decision were substantially higher when automated advice was correct than when it was wrong: pooled odds ratio 10.19, with a 95% confidence interval of 7.43 to 13.98. The authors also report a sensitivity analysis limited to studies from 2020 onward; its estimate was similar, but less precise, at 9.22, with a confidence interval of 2.06 to 41.13.
Evidence spans more than modern generative AI
The review covers controlled and real-world studies of clinician responses to incorrect automated advice. Its findings therefore concern clinical decision support broadly, rather than a single product or a test of one current large language model. The ECG results are a distinct real-world subgroup within that wider evidence base.
The authors say contemporary AI and large language model studies also documented harmful reliance on incorrect outputs, but the studies were too heterogeneous to combine into a pooled estimate. Patient-level adverse events appeared in just one retrospective cohort in the review. The paper therefore does not establish a general rate of patient harm from clinical AI, or show that the ECG estimate applies to generative systems.
“Error-interception capability” is the proposed test
The paper’s central recommendation is to measure whether oversight works as a safety function. “Human oversight should be evaluated as an error-interception capability rather than assumed to ensure safety,” write Zeng, Lei and Li.
That distinction shifts attention from whether a clinician is technically present in a workflow to whether the clinician identifies and resists incorrect advice. The review documents a measurable gap in several types of evidence, but it does not identify one universal cause for missed errors or show which intervention would close that gap.
The authors report no funding for the study and declare no competing interests. Their results leave health systems with a practical measurement problem: how often do clinicians catch erroneous advice in the specific tools and workflows they use, and how often do those errors reach patients?