Weakly Supervised Pneumonia Localization from Chest X-Rays Using Deep Neural Network and Grad-CAM Explanations

AI summary

Overview

• Research area: Medical computer vision / explainable AI — weakly supervised pneumonia classification and localization on chest X-rays.
• Technical level: Intermediate (assumes familiarity with CNNs, transfer learning, and class activation mapping).
• Scope: The paper benchmarks seven ImageNet-pretrained backbones (six CNNs plus a Vision Transformer) on a single public chest X-ray dataset under identical training conditions, and evaluates Grad-CAM heatmaps both qualitatively and with a novel quantitative "Lung Attention Ratio" (LAR) metric.

What This Paper Is About

Pneumonia is commonly diagnosed from chest X-rays, but training a model to point to where the disease is typically requires costly pixel-level annotations such as segmentation masks or bounding boxes. This paper asks whether a model trained only on image-level labels ("normal" vs. "pneumonia") can still produce clinically meaningful heatmaps that highlight the affected lung regions. The authors train and compare seven architectures under the same protocol, then use Grad-CAM to generate those heatmaps and measure whether the model's attention actually falls inside the lungs.

Key Contributions

1. Patient-level data re-splitting. The authors noticed that some patient IDs appeared in both the original training and test sets of the Kermany dataset, which risks data leakage. They merged the original splits and re-partitioned the data by patient into 70% training, 15% validation, and 15% test.
2. Unified benchmark of seven backbones. ResNet-18, ResNet-50, DenseNet-121, EfficientNet-B0, MobileNetV2, MobileNetV3, and ViT-B/16 were trained and evaluated under identical conditions, using focal loss and class-balanced sampling.
3. Consistent Grad-CAM across CNNs and transformers. The paper applies standard Grad-CAM to the last convolutional layer for CNNs, and extends the formulation to ViT-B/16 by taking activations and gradients from the final Linear layer of the last MLP block in the last transformer encoder layer.
4. A quantitative localization metric. Because no pixel-level ground truth is available, the authors introduce the Lung Attention Ratio (LAR) — the share of Grad-CAM activation energy falling inside a coarse lung region of interest — and report it per class for all seven models.

Main Findings

• High classification accuracy across the board: Test accuracy ranged from 96% to 98%. ResNet-18 and EfficientNet-B0 reached the highest test accuracy at 98%, each with a best F1-score of 0.987; ResNet-50 had the lowest test accuracy at 96%.
• Compact models held their own: MobileNet-V3 (4.9M parameters) achieved 97% test accuracy with ROC-AUC 0.9971, PR-AUC 0.9990, and F1 0.987 — matching the ROC-AUC and PR-AUC of ViT (86.2M parameters) and ResNet-18 (11.5M parameters).
• Deeper backbones did not help: ResNet-50 (24.6M parameters) and DenseNet-121 (7.5M parameters) showed marginally lower generalization (96% and 97% test accuracy respectively), which the authors attribute to mild overfitting.
• MobileNet-V3 localized best on pneumonia: It produced the most compact, noise-free heatmaps, with an LAR of 0.693 ± 0.072 for bacterial pneumonia and 0.584 ± 0.280 for viral pneumonia — the highest and most stable values for those classes among the tested models.
• ViT-B/16 separated normal from bacterial pneumonia most sharply: It had the lowest LAR on normal images (0.172 ± 0.038) and a high LAR on bacterial pneumonia (0.605 ± 0.133), though its separation for viral pneumonia was weaker (0.381 ± 0.096), which the authors link to diffuse global self-attention.
• Normal X-rays produced weak, diffuse activations: Heatmaps for normal cases were generally weak and often extended beyond lung boundaries. DenseNet-121 was an exception, showing spurious activation along the left lung region.
• EfficientNet-B0's explanations were less focused: Despite its 98% test accuracy, its Grad-CAM maps were comparatively diffuse and occasionally midline-biased; ResNet-50 showed intermittent off-target hotspots.
• Per-class behavior: On the test set, ResNet-18 reached precision 0.97 / recall 0.96 for normal and 0.99 / 0.99 for pneumonia, while ViT reached 0.95 / 0.95 for normal and 0.98 / 0.98 for pneumonia.

Methodology in Plain English

The authors took a publicly available chest X-ray dataset (Kermany et al.) containing normal and pneumonia images. They found that the original train/test split shared patient IDs, so they rebuilt the split at the patient level: training 70%, validation 15%, test 15%. Table 1 reports the resulting counts — training: 1,114 normal and 2,951 pneumonia; validation: 232 normal and 653 pneumonia; test: 237 normal and 669 pneumonia. (When motivating focal loss, the paper separately cites an imbalance of 404 normal vs. 3,692 pneumonia images, which does not match the Table 1 split counts.)

Every image was resized to 224×224 pixels and, because the originals are grayscale, duplicated into three channels to create a pseudo-RGB input compatible with ImageNet-pretrained backbones. Augmentation included random rotation, horizontal flipping, brightness/contrast adjustment, Gaussian noise, plus ±15° rotations, ±5% affine transformation, 5% brightness-contrast adjustment, CLAHE, gamma correction, motion blur, median blur, and coarse dropout.

Each of the seven pretrained models was given the same custom classification head (fully connected layers, batch normalization, ReLU, dropout). Training used the Adam optimizer with a learning rate of 1×10⁻⁴ and weight decay of 1×10⁻⁴, batch size 32, up to 10 epochs, and early stopping after three consecutive epochs without validation improvement. Three loss functions were considered — cross-entropy, weighted cross-entropy, and focal loss — with focal loss selected for class imbalance, combined with random over-sampling of the minority class. The best checkpoint per model was chosen by validation accuracy and ROC-AUC, then evaluated on the held-out test set using accuracy, ROC-AUC, PR-AUC, and best F1. Experiments ran on PyTorch 2.8.0+cu126 with an NVIDIA T4 GPU with 15 GB VRAM.

For interpretability, Grad-CAM heatmaps were generated for each architecture. To score them without pixel-level labels, the authors defined a coarse lung region of interest from a fixed thoracic anatomical prior that excludes image borders and sub-diaphragmatic regions, kept only the top 20% of activation values to suppress background noise, and computed LAR as the ratio of activation inside the lung ROI to total activation in the image. Evaluation used a fixed representative subset of test images from each class, identical across all architectures.

Why This Matters

Impact on research. The paper shows that a rigorous, leakage-free split can be applied to a widely used benchmark and still yield 96–98% accuracy across architectures, which raises the bar for how comparable results should be reported. It also provides a reproducible, label-free way to sanity-check whether saliency explanations are anatomically plausible, rather than relying only on visual inspection.

Real-world applications:

• Triage in resource-limited settings, where a lightweight model such as MobileNet-V3 (97% test accuracy, 4.9M parameters) could run on edge or mobile hardware.
• Radiologist decision support, where heatmaps give an intuitive visual cue that ties a prediction to radiographic opacities in the middle and lower lung zones.
• Automated screening workflows in hospitals and clinics that need fast normal/pneumonia separation before a human reads the image.
• Educational and audit tools for reviewing which image regions drive a model's output, since activation patterns reveal failure modes such as off-target hotspots.

Industry relevance. The findings point to compact architectures as the practical choice when balancing accuracy, latency, and cost — relevant to medical device vendors, mobile health developers, and cloud triage services. The authors also release source code at https://github.com/kiranshahi/pneumonia-analysis.

Future Directions

• Broader and more diverse datasets. The authors note the study is limited to a single dataset and call for evaluation on larger datasets such as RSNA Pneumonia and NIH ChestX-ray14 to test robustness and generalization.
• Multi-label thoracic disease localization, extending the framework beyond binary pneumonia classification to multiple findings.
• Radiologist reader studies to validate whether the Grad-CAM heatmaps actually help clinicians, rather than only appearing plausible.
• Mobile deployment optimization, so lightweight models such as MobileNet-V3 can be pushed further toward real-time edge and mobile health use.

Target Audience

This paper suits graduate students and researchers in medical image analysis and explainable AI, machine learning engineers building clinical decision-support or triage systems, and radiologists or clinical informatics staff who want a non-mathematical sense of what Grad-CAM explanations can and cannot show. Readers should have basic familiarity with convolutional networks, transfer learning, and classification metrics such as ROC-AUC, PR-AUC, and F1-score.

Authors’ abstract

Chest X-ray imaging is commonly used to diagnose pneumonia, but accurately localizing the pneumonia-affected regions typically requires detailed pixel-level annotations, which are costly and time consuming to obtain. To address this limitation, this study proposes a weakly supervised deep learning framework for pneumonia classification and localization using Gradient-weighted Class Activation Mapping (Grad-CAM). Instead of relying on costly pixel-level annotations, the proposed method utilizes image-level labels to generate clinically meaningful heatmaps that highlight pneumonia-affected regions. Furthermore, we evaluate seven pre-trained deep learning models, including a Vision Transformer, under identical training conditions, using focal loss and patient-wise splits to prevent data leakage. Experimental results suggest that all models achieved high classification accuracy (96--98\%), with ResNet-18 and EfficientNet-B0 showing the best overall performance and MobileNet-V3 providing an efficient lightweight alternative. Grad-CAM heatmap visualizations confirm that the proposed methods focus on clinically relevant lung regions, supporting the use of explainable AI for radiological diagnostics. Overall, this work highlights the potential of weakly supervised, explainable models that enhance transparency and clinical trust in AI-assisted pneumonia screening.