Hallucination Filtering in Radiology Vision-Language Models Using Discrete Semantic Entropy

AI summary

Overview

Research area: Medical computer vision and clinical AI safety — specifically hallucination detection and selective prediction for vision-language models (VLMs) in radiology.

Technical level: Intermediate. The paper is written for a clinical and medical-imaging methods audience; the core idea (sampling multiple answers, grouping them by meaning, measuring disagreement) is intuitive, though familiarity with entropy and VLM APIs helps.

Scope: A retrospective evaluation of whether discrete semantic entropy (DSE) — a black-box, consistency-based uncertainty measure — can filter out questions likely to produce hallucinated answers from GPT-4o and GPT-4.1 on radiologic visual question answering, using 706 image–question pairs drawn from two public datasets.

What This Paper Is About

Vision-language models such as GPT-4o can answer questions about radiologic images, but they sometimes produce plausible-sounding answers that are not grounded in the image — hallucinations — which is dangerous in clinical use. Because these models are proprietary black boxes, methods that require internal token probabilities or extra training are hard to apply. The authors test whether asking the same question many times and measuring how much the meaning of the answers varies (discrete semantic entropy) can identify questions at high risk of hallucination, so those questions can be withheld and accuracy on the remaining answers improved.

Key Contributions

1. First application of discrete semantic entropy to multimodal radiologic question answering. DSE was previously validated for text generation (Farquhar et al., 2024); this work extends it to clinical vision-language tasks, including a related but distinct line of work on vision-amplified semantic entropy.

2. A black-box filtering pipeline requiring no model internals or retraining. The method uses only standard API calls — 15 samples per image–question pair at temperature 1.0 plus pairwise entailment checks performed by the same VLM — making it deployable alongside proprietary models.

3. Quantification of the accuracy–coverage trade-off at two thresholds. Twelve pre-specified comparisons (two models × two datasets × combined dataset, at DSE ≤ 0.6 and DSE ≤ 0.3) show how rejecting more questions raises accuracy on retained questions.

4. Practical feasibility analysis. The authors report latency, parallelization structure, and token cost, and they release code publicly at https://github.com/TruhnLab/VisionSemanticEntropy.

Main Findings

• Low baseline accuracy overall. Across 706 image–question pairs, baseline accuracy (temperature 0.1) was 51.7% for GPT-4o and 54.8% for GPT-4.1. On VQA-Med 2019 the models reached 59.0% and 63.2%; on the RadDataset baseline accuracy was much lower at 34.0% and 34.5%.

• DSE filtering improves accuracy on retained questions. At DSE ≤ 0.6 on the combined set, GPT-4o rose from 51.7% to 62.9% (499 of 706 retained, +14.2%, 95% CI +11.7 to +16.8, p < .001) and GPT-4.1 from 54.8% to 60.4% (626 of 706 retained, +5.6%, 95% CI +4.2 to +7.1, p < .001).

• Stricter threshold, larger gain, fewer answers. At DSE ≤ 0.3 on the combined set, GPT-4o reached 76.3% (334 of 706 retained, +24.6%, 95% CI +20.8 to +28.5, p < .001) and GPT-4.1 reached 63.8% (499 of 706 retained, +13.5%, 95% CI +11.0 to +16.2, p < .001). Only 47.3% of questions were answered by GPT-4o in this setting.

• One comparison lost significance. GPT-4o on the RadDataset at DSE ≤ 0.3 improved from 34.0% to 48.3% (+14.3%, 95% CI −2.7 to +31.5, p = .100) but only 29 questions remained, limiting statistical power. All other improvements in the main table were significant after Bonferroni correction (threshold p < .004; the methods section states the adjusted value as p < .05/12 = .0042).

• Performance varied sharply by question subcategory on VQA-Med 2019. Baseline accuracy ranged from 81.6% (GPT-4o) and 83.2% (GPT-4.1) on modality questions to 13.6% and 12.8% on abnormality questions.

• Rejection rates tracked difficulty. For modality questions at DSE ≤ 0.3, only 8.8% of questions were removed (GPT-4o 11/125; GPT-4.1 10/125) with marginal gains of +2.6% and +2.0%. For GPT-4o, abnormality questions had rejection rates of 78.4% at DSE ≤ 0.6 and over 91.2% at DSE ≤ 0.3. Plane and organ subcategories showed rejection rates of 16.0% and 40.0% at DSE ≤ 0.3 for GPT-4o, with gains of +7.8% and +7.7%.

• Model-specific subcategory effects were inconsistent. In the abnormality subcategory, GPT-4o gained +19.7%, whereas GPT-4.1 gained +3.2% at DSE ≤ 0.6 and lost 3.1% at DSE ≤ 0.3. On the RadDataset, CT accuracy for GPT-4o declined by 15.0% after filtering, with only 8.3% of CT questions retained.

• Angiography and radiography benefited most within the RadDataset. GPT-4.1 CT accuracy rose from 31.7% to 45.2% at DSE ≤ 0.3. Angiography rejection rates at DSE ≤ 0.3 were 69.2% (GPT-4o) and 19.2% (GPT-4.1), with accuracies of 75.0% and 61.9%.

• DSE fails on consistent hallucinations. The paper documents a case where the model confidently and consistently answered "Noncontrast MRI" incorrectly; a low DSE score bypassed the filter.

• Feasibility. Mean single-call latency for GPT-4o was about 3 seconds (standard deviation roughly 2 s); with the generation and the k(k−1) pairwise entailment checks parallelized, total pipeline latency was about 6 seconds. At an assumed average price of $10 per million tokens for GPT-4o, cost per image–question pair was about $0.11 for answer sampling and about $0.61 for entailment checks, totaling roughly $0.72 per question.

Methodology in Plain English

The researchers used two public, de-identified datasets. VQA-Med 2019 supplied 500 radiological images with clinical questions and short-text answers, evenly split into four categories (modality, plane, organ, abnormality — 125 pairs each). The RadDataset supplied 206 clinical 2D images with ground-truth diagnoses: 60 CT, 60 MR, 60 radiographs, and 26 angiograms. Together these yielded 706 image–question pairs.

Two OpenAI models accessed through the Microsoft Azure API were tested: GPT-4o (version 2024-05-13) and GPT-4.1 (version 2025-04-14). For each pair, the model produced one answer at temperature 0.1, which defined the baseline accuracy, and 15 answers at temperature 1.0, which were used to compute uncertainty. A medical student (trained by two board-certified radiologists) reviewed every generated answer against the reference, with uncertain cases escalated to the radiologists; an answer counted as correct if its clinical meaning matched the reference, even if worded differently.

To compute DSE, the same model performed a mutual entailment check on every pair of its own 15 answers. Answers were placed in the same semantic cluster only if mutual entailment was confirmed, so that paraphrases of the same meaning would not inflate the entropy score. The relative frequency of each cluster was computed, and entropy was calculated as the negative sum of those frequencies times their base-10 logarithm. If all 15 answers formed one cluster, DSE was zero; the maximum possible value with 15 responses is log10(15) ≈ 1.18, when every answer forms its own cluster.

The filtering step is selective prediction: questions whose DSE exceeded a threshold were discarded, and accuracy was recalculated on the rest. Two thresholds were used — 0.6 and 0.3, corresponding to one half and one quarter of the maximum attainable DSE. Significance was assessed with a two-sided bootstrap test using 100,000 iterations, with 95% confidence intervals from the percentile bootstrap method and a Bonferroni-corrected threshold because twelve comparisons were pre-specified. The study was reported according to the CLAIM checklist; institutional review board approval and informed consent were waived, and no studies were excluded.

Why This Matters

Impact on research. The paper shows that a text-domain uncertainty method transfers to clinical multimodal tasks and provides a concrete, reproducible protocol (public code) for hallucination filtering in black-box VLMs. It also documents the ceiling of consistency-based methods: DSE measures semantic consistency, not factual correctness, so a confidently repeated error passes the filter. This sharpens the research agenda toward complementary uncertainty signals and curated answer retrieval.

Real-world applications:

• Diagnostic decision support: A stricter DSE threshold (e.g., 0.3) could withhold uncertain VLM answers so clinicians are not shown unreliable suggestions for complex interpretive questions.
• Human-in-the-loop screening: A more lenient threshold (e.g., 0.6) preserves sensitivity, keeping more questions answered where missed information is a greater concern than a wrong suggestion.
• PACS or reporting-system integration: DSE runs as a lightweight wrapper on standard API calls and can attach an interpretable uncertainty score to each query or withhold unstable outputs.
• Question triage in clinical query pipelines: Because rejection rates tracked question difficulty (low for modality, extremely high for abnormality), DSE can route easy questions to automated answering and hard ones to humans.

Industry relevance. The method requires no access to model internals, no fine-tuning, and no additional training data, so vendors of proprietary VLMs need not cooperate; the reported cost of roughly $0.72 per question and a latency of about 6 seconds under parallelization make it plausible for routine deployment. Thresholds are model-dependent and must be calibrated per model and per clinical application.

Future Directions

• Addressing confident hallucinations. Combining DSE with other signals — linguistic uncertainty cues, model calibration where available, external clinical context, or cross-model consistency checks — to catch errors the model repeats consistently.
• Replacing self-entailment with an external entailment model. Clustering currently depends on the same VLM being evaluated, so errors or biases in its judgment of semantic equivalence could distort DSE; a clinically validated external entailment model might be more reliable.
• Extending from 2D key slices to volumetric workflows. The current evaluation uses manually selected key slices from CT, MR, radiography, and angiography, which the authors describe as a best-case estimate; integration with automatic key-slice selection or VLMs that natively read 3D stacks is proposed.
• Prospective clinical validation and user studies. Open questions include whether high rejection rates increase clinician trust or make the system impractical, whether selective answering promotes over-reliance on remaining outputs, and how DSE-filtered outputs affect workflow, diagnostic confidence, efficiency, and patient outcomes.
• Broader generalizability testing. Validation across more imaging modalities, pathologies, patient populations, and query types, plus per-model threshold calibration, is needed to transfer the approach to other VLMs.

Target Audience

Radiologists and clinical AI researchers evaluating VLM safety; medical imaging and computer vision researchers working on uncertainty quantification and hallucination detection; clinical informatics and health-IT teams considering deployment of black-box VLMs; and machine learning practitioners interested in consistency-based, training-free uncertainty estimation for multimodal models.

Authors’ abstract

To determine whether using discrete semantic entropy (DSE) to reject questions likely to generate hallucinations can improve the accuracy of black-box vision-language models (VLMs) in radiologic image based visual question answering (VQA). This retrospective study evaluated DSE using two publicly available, de-identified datasets: the VQA-Med 2019 benchmark (500 images with clinical questions and short-text answers) and a diagnostic radiology dataset (206 cases: 60 computed tomography scans, 60 magnetic resonance images, 60 radiographs, 26 angiograms) with corresponding ground-truth diagnoses. GPT-4o and GPT-4.1 (Generative Pretrained Transformer; OpenAI) answered each question 15 times using a temperature of 1.0. Baseline accuracy was determined using low-temperature answers (temperature 0.1). Meaning-equivalent responses were grouped using bidirectional entailment checks, and DSE was computed from the relative frequencies of the resulting semantic clusters. Accuracy was recalculated after excluding questions with DSE &gt; 0.6 or &gt; 0.3. p-values and 95% confidence intervals were obtained using bootstrap resampling and a Bonferroni-corrected threshold of p &lt; .004 for statistical significance. Across 706 image-question pairs, baseline accuracy was 51.7% for GPT-4o and 54.8% for GPT-4.1. After filtering out high-entropy questions (DSE &gt; 0.3), accuracy on the remaining questions was 76.3% (retained questions: 334/706) for GPT-4o and 63.8% (retained questions: 499/706) for GPT-4.1 (both p &lt; .001). Accuracy gains were observed across both datasets and largely remained statistically significant after Bonferroni correction. DSE enables reliable hallucination detection in black-box VLMs by quantifying semantic inconsistency. This method significantly improves diagnostic answer accuracy and offers a filtering strategy for clinical VLM applications.