XAI-MeD: Explainable Knowledge Guided Neuro-Symbolic Framework for Domain Generalization and Rare Class Detection in Medical Imaging

AI summary

Overview

• Research area: Medical artificial intelligence, specifically neuro-symbolic learning applied to medical imaging for domain generalization, rare-class detection, and clinical explainability.
• Technical level: Advanced. The paper combines a Vision Transformer / CNN neural branch with a formal symbolic rule engine, defines two custom selection metrics (Entropy Imbalance Gain and a Rare-Class Gini index), and builds a decision tree over ten classifiers. The writing assumes familiarity with domain generalization benchmarks, mixture-of-experts routing, and propositional logic notation.
• Scope: The paper proposes and evaluates XAI-MeD, a framework that fuses expert-encoded clinical rules with deep learning models and uses an LLM (GPT-4) to produce clinically worded explanations, validated on seizure onset zone localization from rs-fMRI and diabetic retinopathy grading from retinal fundus images.

What This Paper Is About

Deep learning models in medical imaging are typically accurate on the data they were trained on but degrade when deployed at a different hospital, on different scanners, or on rare diseases that appear infrequently in training data, and they offer little explanation that a clinician can act on. This paper builds a framework that keeps the deep model but adds a parallel branch of machine-checkable clinical rules, then routes each case to whichever branch (or sequence of branches) is best suited to it. The goal is to make predictions simultaneously more robust across institutions, more sensitive to rare conditions, and more transparent in language clinicians already use.

Key Contributions

1. A unified neuro-symbolic architecture with two branches — a deep learning branch (f_DL: X → Y) and an Expert Knowledge Processor (f_KL: F* → Y) that maps a structured, human-interpretable knowledge vector F* ∈ ℝ^m to the output space — fused by the Expert Knowledge and Supervised AI Integration (EKSAII) algorithm and passed to GPT-4 for explanation generation.
2. A three-stage knowledge engineering formalism that expresses expert knowledge as atomic propositions, combines them into class-specific logical rules of the form κ_ClassX = (p₁ ∨ ¬p₂) ∧ (p₃ ∧ (p₄ ∨ ¬p₅)) ∧ (¬p₆ ∨ (p₆ ∧ p₇)), and implements each proposition as a graded satisfaction score s_i = K_X(p_i; y), with the class rule satisfaction computed as a weighted sum S_ClassX = Σ w_i s_i. The same formalism is mapped across DR, SOZ, and cardiac proposition sets in Table 1.
3. Two metrics for classifier selection: the Entropy Imbalance Gain (EIG), derived from local density λ(x_i) over K-nearest intraclass neighbors and class entropy θ_r, defined as EIG(M_d) = η_R − η_Md; and the Gini index Gini(s) = 1 − Σ p_i², used to decide when to cascade another classifier onto a high-variability partition.
4. A Hunt-inspired adaptive routing mechanism that builds a decision tree of class-specialized expert models, instantiated with ten binary one-vs-rest classifiers for the five-class DR task. The algorithm also incorporates a rare-class check: if the selected classifier's label set contains the rare class c_r, purity is measured and the process restarts if Gini(s) > τ_g.

Main Findings

• Abstract-level claims: The abstract reports performance across 6 multicenter datasets, with 6% gains in cross-domain generalization and a 10% improved rare-class F1 score over state-of-the-art deep learning baselines. The introduction and conclusion instead state experiments on ten multicenter datasets, with the same 10% rare-class F1 improvement and 6% cross-domain gain. This discrepancy between 6 and ten datasets is unresolved in the text.
• SOZ localization: XAI-MeD reached 84.6% accuracy and 89.7% sensitivity, versus 46.1% / 48.9% for the standalone 2D CNN DL branch and 75.0% / 79.5% for the knowledge-only baseline (EPIK). Effort dropped from over 110 independent components reviewed by an expert to 18, described as an 84.2% reduction.
• Branch routing on SOZ: EIG(EKIE) = 0.22 versus EIG(DL) = 0.027, so the EKIE (expert knowledge) branch was chosen as the primary classifier, with the DL branch refining its partitions by Gini impurity. SOZ independent components are noted as rare, at approximately 5 per subject, prompting SMOTE on the 4-D feature space.
• Cross-center generalization on SOZ: A model trained at Phoenix Child Health Center (PCH) was tested without fine-tuning on data from the University of North Carolina (UNC), holding a statistically equivalent accuracy of 87.5%. Even as the DL branch's noise-classification accuracy fell from 80% to 70% on the new domain, the EKIE branch compensated for the shift. Explanations generated for each result were verified by medical experts.
• DR rare-class detection: The standalone ViT branch scored F1-scores of 45.2% and 51.8% on severe DR grades 3 and 4; XAI-MeD raised these to 56.01% (a +10.8% gain) and 62.4% (a +10.6% gain).
• Multi-domain generalization (MDG), trained on three domains and tested on the held-out one: XAI-MeD (ViT + EKIE fusion) averaged 67.95% accuracy, above SPSD-ViT (65.5%), EKIE alone (63.7%), Fishr (62.2%), and the standalone DeiT-Small DL branch (61.2%). Per-dataset XAI-MeD scores were 53.1 (Aptos), 74.8 (Eyepacs), 68.3 (Messidor), and 75.6 (Messidor 2).
• Single-domain generalization (SDG): XAI-MeD beat the reported best baseline in three of four settings — trained on APTOS it reached 59.9% versus 58.6% (SD-ViT); trained on MESSIDOR it reached 67.1% versus 55.9% (SPSD-ViT); trained on MESSIDOR2 it reached 65.5% versus 62.1% (SPSD-ViT). Trained on EYEPACS it reached 61.7%, below the 62.5% SPSD-ViT baseline.
• Ablation (trained on APTOS, tested on EyePACS / Messidor-1 / Messidor-2): Neural-only ViT scored 66.6 / 46.4 / 48.9; symbolic-only lesion features scored 66.4 / 49.6 / 53.9; non-weighted neural + symbolic fusion scored 72.8 / 50.6 / 54.3, reported as the highest average; weighted fusion scored 67.4 / 49.6 / 53.9. The text describes these first-column values as averages, which does not match the table layout.
• XGBoost was selected for the knowledge machines based on Ablation Study 1, and lesion features such as hemorrhages and exudates were produced by a YOLOv11 detector.
• Decision tree structure for DR (84% accuracy): an initial triage by M_d⁰ (ViT) for "0 vs. Not 0", alternating with ViT classifiers (M_d¹, M_d³) for general pattern recognition and knowledge classifiers (M_k⁴, M_k²) for clinically salient distinctions.
• Not reported: the number of patients or subjects in any dataset, the exact train/test sample counts, confidence intervals or significance tests other than the "statistically equivalent" claim on the UNC transfer, and any cardiac experiment, despite "Heart (Cardiac Function Assessment)" appearing as a domain column in Table 1. The abstract's mention of "four challenging tasks" is followed by only two listed items, and only two tasks (SOZ, DR) are described in the body.
• Naming inconsistencies in the paper: Algorithm 1 is titled "NeuroGuard: Knowledge-Guided Sample Selection and Training" while the surrounding text calls it EKSAII; the SOZ figure is labeled "DeepXSOZ"; and the symbolic branch is variously called EKIE and KL.

Methodology in Plain English

The researchers start by asking clinical experts to write down what they actually look for. Those observations become atomic propositions — statements like "lesion is present" or "frequency is above threshold" — which are then combined with logical connectors into rules that describe each disease class. Because a rule is rarely simply true or false from an image, each proposition gets a graded satisfaction score, and the rule's overall score is a weighted sum of those scores, so a clinician can inspect which piece of knowledge contributed how much.

In parallel, a standard deep model (a 2D CNN for the seizure task, ViT-based models such as DeiT for the retina task) is trained normally on the images. The framework then has two families of classifiers to choose from, and it needs a principled way to pick which one should handle which part of the data. That is what EKSAII does. It computes an Entropy Imbalance Gain for each candidate classifier, measuring how much that classifier reduces the unevenness of the class distribution, especially for the rare class. It picks the classifier with the highest gain. If that classifier's partition of the data is still impure — measured by the Gini index — it cascades another classifier on top. If two classifiers tie, it compares their confidence scores against a dependability threshold.

Because each classifier handles one class versus the rest (ten binary classifiers for five DR grades), the result is a decision tree whose nodes alternate between deep and knowledge-based models. The final diagnosis, together with the symbolic knowledge features, is passed to GPT-4, which writes an explanation in clinical language alongside a spatial localization of the relevant features.

Why This Matters

Impact on research. The paper argues that the dominant approach of fusing symbolic reasoning post-hoc, as an auxiliary feature or an explanation layer, misses the benefit of parallel specialization. Its central methodological claim is that expert knowledge should participate in classifier selection itself, guided by explicit metrics rather than learned data statistics, and that this is what produces robustness under distribution shift. The ablation result that non-weighted neural + symbolic fusion outperformed either branch alone supports the argument that clinical rules can act as regularizers. The framework is also presented as domain-agnostic, with the same proposition formalism mapped across three clinical domains.

Real-world applications:

• Epilepsy surgery planning: reducing the number of independent components a clinician must manually review from over 110 to 18 for seizure onset zone localization, while maintaining 84.6% accuracy and 89.7% sensitivity.
• Ophthalmology screening: five-class diabetic retinopathy grading at 84% accuracy, with particular value in flagging the severe grades that standalone ViT models miss — grade 3 F1 rose from 45.2% to 56.01% and grade 4 from 51.8% to 62.4%.
• Cross-institution deployment: the PCH-to-UNC transfer result, where accuracy held at 87.5% even as the neural branch's noise classification degraded from 80% to 70%, speaks to sites that cannot retrain or fine-tune on local data.
• Clinical decision support with auditable reasoning: because each classification is accompanied by a text explanation grounded in named propositions and verified by medical experts in the SOZ setting, the output can be reviewed and challenged rather than accepted as a black-box score.

Industry relevance. The framework targets the practical blockers that keep medical imaging models from regulatory and clinical acceptance: generalization failure across scanners and protocols, underperformance on rare conditions that carry the highest clinical risk, and explanations that clinicians cannot map onto pathology. Its explicit dependence on auxiliary detectors (YOLO, U-Net) and expert-authored rules, however, means deployment cost includes annotation and knowledge-engineering labor, not just compute.

Future Directions

1. Removing the auxiliary-model bottleneck. The authors state that YOLO lesion detectors and U-Net anatomical segmenters are required to extract clinically meaningful intermediate representations, and that these depend on manually annotated data that is expensive and not available in all settings. They propose weakly supervised or self-supervised lesion discovery, and fully end-to-end architectures that learn interpretable structure without separate detectors.
2. Expanding beyond a fixed rule set. The symbolic module cannot fully capture rare or edge scenarios absent from the training corpus, so the authors propose rule induction from large medical corpora using large language models.
3. Resolving the reported evaluation scope. Because the abstract cites 6 multicenter datasets while the introduction and conclusion cite ten, and because the abstract refers to "four challenging tasks" while the body describes two, the reported scope needs to be reconciled, with per-dataset subject counts and splits made explicit.
4. Testing the third domain in Table 1. Cardiac function assessment propositions (abnormal wall movement, high heart rate variability, ST elevation, arrhythmia, QRS width abnormality, low ejection fraction, valve dysfunction) are formalized but no cardiac results are reported, leaving that generalization untested.

Target Audience

This paper is most useful to researchers working on neuro-symbolic AI, domain generalization, and trustworthy medical machine learning, particularly those interested in how symbolic constraints can participate in model selection rather than only in post-hoc explanation. It also speaks to clinical AI engineers and informaticists who need to deploy models across multiple hospital sites and rare disease classes, and to clinician-scientists willing to formalize their diagnostic reasoning as propositions and rules. Readers seeking an accessible introduction to medical AI will likely find the formal notation and the density of unexplained metric definitions (EIG, Gini thresholds, dependability thresholds) demanding; readers looking for a production-ready system will note the framework's reliance on auxiliary annotation-heavy detectors and expert rule authoring.

Authors’ abstract

Explainability domain generalization and rare class reliability are critical challenges in medical AI where deep models often fail under real world distribution shifts and exhibit bias against infrequent clinical conditions This paper introduces XAIMeD an explainable medical AI framework that integrates clinically accurate expert knowledge into deep learning through a unified neuro symbolic architecture XAIMeD is designed to improve robustness under distribution shift enhance rare class sensitivity and deliver transparent clinically aligned interpretations The framework encodes clinical expertise as logical connectives over atomic medical propositions transforming them into machine checkable class specific rules Their diagnostic utility is quantified through weighted feature satisfaction scores enabling a symbolic reasoning branch that complements neural predictions A confidence weighted fusion integrates symbolic and deep outputs while a Hunt inspired adaptive routing mechanism guided by Entropy Imbalance Gain EIG and Rare Class Gini mitigates class imbalance high intra class variability and uncertainty We evaluate XAIMeD across diverse modalities on four challenging tasks i Seizure Onset Zone SOZ localization from rs fMRI ii Diabetic Retinopathy grading across 6 multicenter datasets demonstrate substantial performance improvements including 6 percent gains in cross domain generalization and a 10 percent improved rare class F1 score far outperforming state of the art deep learning baselines Ablation studies confirm that the clinically grounded symbolic components act as effective regularizers ensuring robustness to distribution shifts XAIMeD thus provides a principled clinically faithful and interpretable approach to multimodal medical AI.