The Pulse
Medscape Says Clarus Beat Top Published Competitor Scores on Two Health AI Benchmarks
Medscape reported Clarus scores of 88.6% on NOHARM v2 and 85.0% on HealthBench Professional, exceeding the three highest published competitor scores on each benchmark. The company also described ClinVal, a clinician-developed framework that
AI.info Team · Written with AI · fact-checked ·
On Nov. 4, Medscape reported scores of 88.6% for its Clarus model on NOHARM v2 and 85.0% on HealthBench Professional. The company said the results reflect an approach that combines model performance with clinical context and trusted medical information for healthcare professionals. Medscape also shared details of ClinVal, its proprietary evaluation framework, for the first time. The release describes ClinVal as a methodology for assessing the quality of AI-generated answers and guiding further development of Medscape’s AI answer engine.
Medscape says Clarus scored above the three highest published competitor scores
Medscape describes NOHARM v2 as a benchmark that examines the frequency and severity of potentially harmful errors in AI-generated medical recommendations. It says Clarus scored 88.6% on the benchmark, exceeding the three highest reported competitor scores: 86.2%, 84.5% and 80.0%. Medscape describes HealthBench Professional as assessing AI responses to health-related scenarios against physician-defined standards for answer quality. It says Clarus scored 85.0% on that benchmark, surpassing the three highest published competitor scores of 69.2%, 65.6% and 65.3%.
Medscape described NOHARM v2 and HealthBench Professional as leading benchmarks for clinical AI safety and performance. The release says the results reflect the company’s focus on clinical AI quality and safety. Joshua Conrad, PharmD, Group VP Editorial & Content Strategy at Medscape, said: “These results reflect the rapid progress we're making in developing AI for healthcare. We're laser-focused on systematic testing and continued refinement, with the goal of delivering meaningful value to healthcare professionals in real-world practice.”
Medscape says ClinVal uses 16 criteria to review answers
Medscape describes ClinVal as a clinician-developed framework for systematically assessing and improving the answers generated by its AI models. It says the framework evaluates answers against an evidence-based set of 16 criteria across dimensions including safety, accuracy, relevance and traceability. Medscape clinicians established the criteria and refined them in collaboration with AI engineering teams. The review process combines automated AI assessments with clinician review, bringing clinical expertise into the assessment of answer quality.
Medscape says the framework supports an ongoing cycle of evaluation and improvement. Findings help Medscape identify opportunities to improve the product, and the company re-evaluates answers after changes to assess their impact. Medscape says the repeatable process lets it measure answer quality throughout development, rather than relying only on periodic external benchmark results. The release says this process is intended to measure quality throughout the ongoing development of Medscape’s AI technology.
Medscape has shared ClinVal with external physicians for feedback
Medscape has shared ClinVal with external physicians to gather additional feedback on the framework’s evaluation methodology and clinical relevance. Conrad said: “Strong benchmark results demonstrate what our AI can achieve today, but clinical AI quality requires an ongoing commitment to improvement. With ClinVal, we've established a process that combines AI-driven evaluation with clinical expertise to identify where our answers can be better, make improvements, and measure the results. This continuous cycle is fundamental to how we're developing Medscape's AI models.”