# Microsoft AI Scores 83% on Complex Cases, Versus 36% for Doctors

Published 2026-10-23 · AI.info Team · AI.info

> Microsoft’s MAI-DxO diagnostic system scored 83% on complex New England Journal of Medicine cases, compared with 36% for physicians using reference tools, the company says. The Science study tested the system on a step-by-step benchmark, while a planned hospital study will examine its performance with patients.

## 83% on complex cases, against 36% for physicians

[Microsoft](https://microsoft.ai/news/building-medical-superintelligence/)’s AI diagnostic system reached 83% accuracy on complex medical cases, compared with 36% for physicians who could consult textbooks and the internet, the company said October 22. The results appeared in *Science* in a study of sequential diagnosis: a clinician or AI system starts with limited information, then decides what to ask or test before settling on a diagnosis. [Microsoft](https://ai.info/companies/microsoft)’s system, called MAI-DxO, also scored 80% on more typical emergency-department cases, compared with 47% for physicians.

The comparisons are striking, but they describe performance on a benchmark, not outcomes for patients receiving care. MAI-DxO was about 2.3 times as accurate as physicians with reference tools on the complex cases, and about 3.7 times as accurate as physicians working without them, Microsoft says. On the emergency-department cases, its accuracy was more than 1.7 times the physician result.

## MAI-DxO asks what to do next

The benchmark, called the Sequential Diagnosis Benchmark, includes 304 complex cases drawn from *The New England Journal of Medicine* and a second set of more typical emergency-department presentations from MIMIC-IV-ED. Rather than show all case details at once, it gives the system an initial summary and lets it request additional information in stages. The evaluation tracks accuracy, evidence requested and estimated testing costs, so it measures the route to a diagnosis as well as the final answer.

MAI-DxO organizes multiple rounds of proposed diagnoses, questions and tests. The system weighs each test’s likely value against its cost and updates its assessment as new information arrives. Microsoft describes the benchmark this way: “Each becomes an interactive case in which AI models and/or physicians start with limited information and choose what to request next.”

A cost-focused configuration also reached 81% accuracy on the complex cases at less than half the cost of comparative systems, the company reports. Its broader claim is that no other tested human or AI approach was both more accurate and less costly across the two datasets. Those cost estimates are part of the benchmark, not evidence of savings already achieved in a hospital.

## The benchmark has limits built into its design

SDBench models a step-by-step workup, but its cases are drawn from existing records rather than conversations with patients in live clinical settings. The benchmark’s gatekeeper returns a real finding when one is in the case record; when it is absent, the gatekeeper can provide a synthetic result. That design choice is documented on the [SDBench site](https://microsoft.github.io/sdbench/) and matters when interpreting performance: some requested information may be generated for the simulation rather than drawn from the original case.

Access to the underlying case sets also has conditions. The NEJM cases must be licensed from NEJM, while the MIMIC dataset requires credentialing, training and acceptance of its data-use agreement. Microsoft is releasing benchmark and evaluation code for select NEJM cases so researchers can reproduce results and test other models; the benchmark site distinguishes those licensed clinical cases from five synthetic cases provided for open, access-free evaluation.

The study therefore supports a narrow conclusion: MAI-DxO performed better than the physician comparison groups on the cases and under the conditions tested. It does not establish that the system outperforms specialists, clinical teams or doctors working with patients in ordinary practice. Nor does a higher benchmark score by itself show that using the tool improves patient health.

## A hospital study is the next test

A new partnership with Beth Israel Deaconess Medical Center in Boston will evaluate clinical reasoning across hospital settings. In the planned prospective study, patients will interact directly with the system, and its performance will be measured against physicians’ independent assessments. That test addresses a different question from SDBench: whether the approach works when people, clinical workflows and real-world care are involved.

For now, the 83% result belongs to a controlled evaluation of written cases. The next concrete measure will come from the hospital study Microsoft says it plans to conduct, where the system’s recommendations can be assessed alongside independent physician judgments.

## Sources

1. Original source [Microsoft AI](https://microsoft.ai/news/building-medical-superintelligence/) microsoft.ai

Other pages read

2. [Microsoft SDBench](https://microsoft.github.io/sdbench/) microsoft.github.io

## Companies in this story

- [Microsoft](https://ai.info/companies/microsoft)
- [Microsoft Research](https://ai.info/companies/microsoft-research)

## Related stories

- [Microsoft Completes Pretraining of Its 2.8T-Parameter MAI-Base-2](https://ai.info/news/microsoft-completes-pretraining-mai-base-2)
- [Google’s AMIE Included the Final Diagnosis in 90% of Clinic Cases](https://ai.info/news/google-amie-final-diagnosis-clinic-study)
- [RADAR Scores 0.913 AUC Across 146 Abdominal CT Findings](https://ai.info/news/radar-scores-0913-auc-across-146-abdominal-ct-findings)
- [Microsoft Puts MAI Image 2.7 Editing Model Into Preview](https://ai.info/news/microsoft-mai-image-2-7-editing-preview)
- [Anthropic Uses Claude to Prove 239 Tests Across 123 Libraries](https://ai.info/news/anthropic-claude-formal-verification-123-libraries)

## Explore

[More articles](https://ai.info/news)

---

Canonical: https://ai.info/news/microsoft-ai-diagnostic-system-science-study
