HexMIL: Hierarchical Attention MIL for Ante-Hoc Explainable Detection of AI-Manipulated CT Volumes

AI summary

Overview

Research area: Computer vision for medical image forensics — specifically the detection and localization of AI-manipulated (deepfake) 3D CT volumes, combining multiple instance learning (MIL), attention-based explainability (XAI), and cross-generator generalization.

Technical level: Advanced. The paper assumes familiarity with Gated Attention MIL, GANs and diffusion models, out-of-domain evaluation protocols, and volumetric localization metrics (IoU, Pointing Game, Pixel AUC).

Scope (one sentence): The paper introduces HexMIL, a mask-free, natively interpretable detector that hierarchically decomposes CT volumes into patches and slices to detect AI-manipulated sub-volumes and localize them through its own attention weights.

What This Paper Is About

Detectors for medical deepfakes — CT scans in which malignant nodules have been injected or removed by generative models — currently fail in two ways: they do not generalize to generative architectures they were not trained on, and they operate as unexplainable black boxes. HexMIL addresses both problems at once by learning a two-level hierarchy of attention (over patches within a slice, then over slices within a volume), using only binary volume-level labels ("pristine" or "tampered"), with segmentation masks reserved exclusively for evaluation.

Key Contributions

1. A unified framework for robust and interpretable 3D forensics. HexMIL is presented as the first medical deepfake detector to jointly address out-of-domain generalization to unseen generative architectures while being intrinsically interpretable.

2. Hierarchical mask-free training with intrinsic 3D spatial attribution. No pixel-level supervision is required, yet the model still detects and localizes tampered regions, with masks used only for evaluation.

3. Attention-based MIL for native 3D explainability. The first application, per the authors, of attention-based multiple instance learning for explainability in 3D medical deepfake detection. The patch- and slice-level gated attention weights are combined into a full-resolution 3D attention volume that localizes the manipulated sub-region; unlike Grad-CAM or Grad-CAM++, these weights are the exact forward computation driving classification rather than a post-hoc gradient approximation.

4. State-of-the-art cross-generator performance. HexMIL outperforms all baselines in out-of-domain detection and localization, and its intrinsic attention volumes beat post-hoc Grad-CAM and Grad-CAM++ on both Pointing Game and IoU under identical supervision conditions (as reported in Figure 3 of the paper).

Main Findings

• Out-of-domain classification: HexMIL reaches an average AUC of 92.0 and F1 of 90.2, compared with the strongest baseline averages of 82.9 AUC and 80.8 F1 — absolute gains of +9.1 AUC and +9.4 F1 (approximately +10.9% and +11.6% relative). It dominates all transfer directions in AUC and nearly all in F1.

• Baseline comparison — ManTraNet: Leads the competitors in average AUC at 82.9, but its much lower average F1 of 73.7 indicates strong ranking ability offset by weak calibration under distribution shift.

• Baseline comparison — D³, TruFor, FreqNet: D³ attains the second-highest average AUC of 80.7 but varies across transfer directions; TruFor averages 80.3 AUC and FreqNet 62.9 AUC, showing that specialized noise-sensitive or high-frequency forensic detectors struggle when appearance and forensic statistics change simultaneously.

• Single-level attention is insufficient: ResNet50-ABMIL and ViT-ABMIL, which apply gated attention slice-by-slice without a volume-level aggregation stage, perform much worse, with an average AUC close to 65.

• Native 3D architectures underperform: 3D CNNs (R3D-18, DenseNet121-3D) and Video Vision Transformers (ViViT) reach at most 72.9% AUC. The authors attribute this to the spatial sparsity of CT manipulations, which occupy only a small fraction of a scan.

• Out-of-domain localization: Despite never being trained for localization, HexMIL achieves the best average out-of-domain localization with an IoU of 42.4 and a Pointing Game score of 70.6, leading in most transfer cells. MVSS-Net and ManTraNet are nearly tied on average IoU (38.1 vs 38.0), with ManTraNet recording the highest baseline PG of 67.5; TruFor averages 36.6 IoU and 65.3 PG.

• Ante-hoc versus post-hoc: On the same HexMIL model, the authors report that the native attention maps outperform Grad-CAM and Grad-CAM++ on IoU, Pixel AUC, and Pointing Game (Figure 3); the numeral values for that comparison are not included in the truncated content provided.

• Interpretation of the gap: Previous methods tend to be strong along one axis at a time (classification, calibration, overlap, or pointing), whereas HexMIL is consistently strong across all axes — a result the authors attribute to its hierarchical architecture rather than a 2D slice-level evaluation followed by pooling.

Methodology in Plain English

HexMIL is trained in two sequential stages.

Stage 1 — SliceMIL (slice encoder). Each axial CT slice is tiled into overlapping patches of size P = 64 with stride S = 32. A shared ResNet-50 encoder maps every patch to a D = 512-dimensional feature. A Gated Attention module (with internal dimension L = 128) then pools the patches into a single slice representation, where each patch weight is directly readable as its forensic relevance. A two-layer fully-connected head produces a binary logit for the slice. This stage is trained with binary cross-entropy (Li_c slc), using slices sampled from pristine volumes and slices intersecting the manipulated sub-volume, identified from nodule axial coordinates available as dataset metadata. No spatial supervision is used inside the slice.

Stage 2 — HexMIL (volume aggregation). The trained SliceMIL is frozen and used to encode every slice in a volume of K = 32 consecutive slices. A fixed sinusoidal positional encoding is added to each slice representation so that axial ordering is preserved (SliceMIL alone processes slices independently and conveys no position information). A second, independent Gated Attention module (internal dimension L′ = 256) produces inter-slice weights, and the weighted sum of position-aware slice features is classified by a fully-connected head. Only the volume-level parameters are optimized in this stage (Li_V), which the authors say decouples patch-level forensic feature learning from inter-slice evidence aggregation and prevents catastrophic forgetting.

Inference. Because real scans may have K′ slices, the volume is split into T = int(K′/K) non-overlapping sub-volumes (with padding if K′ < K or K′ is not divisible by K), each receives a probability, and the volume score is the maximum over sub-volumes.

Explainability module. The inter-slice weights β_k and the intra-slice patch weights α_k are combined multiplicatively into a 3D attention volume: slices below τ_β = 0.1 are gated out first, remaining patch maps are reshaped and bilinearly upsampled to slice resolution, then multiplied by the gated slice weight. Sub-volume maps are concatenated along the slice axis, smoothed with a Gaussian kernel (σ = 0.03 · max(K, H, W)) and min-max normalized globally so scores are comparable across the whole scan. Thresholding at τ_3D = 0.65 selects a fraction of the most attended voxels (a τ_3D in (0,1) selects the top (1 − τ_3D) · 100%), from which the tightest axis-aligned 3D bounding box is extracted.

Training setup. Both stages run for 100 epochs with AdamW (lr = 10⁻⁴, wd = 10⁻⁵) on a single NVIDIA A6000 GPU; augmentation is random horizontal and vertical flips. The authors state the experiments are fully reproducible.

Why This Matters

Impact on research. The paper reframes detection and explanation as the same computation: the attention weights that aggregate instances are the classification function itself, not a retrofit onto a frozen model. The authors contrast this with work showing that some saliency methods (citing Adebayo et al.) produce plausible-looking but semantically meaningless maps that survive model-weight randomization. The paper also extends attention-based MIL — previously dominated by 2D computational pathology — to 3D CT volumes with a second aggregation level and a two-stage training protocol.

Real-world applications:

• Hospital and PACS integrity: Detecting malicious injection or removal of lesions in clinical archives that rely on legacy Picture Archiving and Communication Systems with weak security mechanisms.
• Diagnostic safety: Flagging tampered scans before they influence a treatment plan, given that CT-GAN-style manipulations have been shown to fool radiologists and commercial CAD systems.
• Forensic and insurance auditing: Providing a spatial bounding box as evidence for where a scan was altered, without needing expensive voxel-level annotations.
• Dataset provenance and curation: Screening large medical image collections, including data used to train downstream diagnostic AI, for synthetic or edited content.

Industry relevance. The method requires only binary volume-level labels, which is far cheaper than the dense pixel-level annotation that methods such as MVSS-Net require for 3D volumes. Combined with its cross-generator robustness, that makes it practical for medical imaging vendors, radiology AI developers, and clinical IT security teams who cannot annotate manipulation masks at scale.

Future Directions

• Extending beyond CT. The paper positions cross-generator generalization as the core test; extending the hierarchy to other modalities, such as the MRI scans mentioned in related work, is a natural next question that the paper does not evaluate.
• Robustness of the XAI thresholds. The explainability module depends on several fixed hyperparameters (τ_β = 0.1, τ_3D = 0.65, Gaussian σ = 0.03 · max(K, H, W)). How sensitive localization quality is to these settings, and whether they should adapt per scan, is not reported.
• Ante-hoc versus post-hoc quantification. The paper claims intrinsic attention beats Grad-CAM and Grad-CAM++ under identical supervision, but the numeric values of that comparison in the truncated content provided are not available; a fuller head-to-head across all transfer directions would sharpen the claim.
• Removal versus injection attacks. Related work notes that removal attacks are harder because the inpainted region is statistically close to healthy tissue; the paper reports aggregate averages but does not, in the content provided, break out performance by attack type.
• The K′ < K and non-divisible cases. Inference relies on padding when K′ < K or (K′ ≠ 0 mod K); the effect of this padding on localization is not reported in the supplied text.

Target Audience

Researchers and practitioners in medical image forensics, explainable AI, and multiple instance learning; developers of clinical AI and PACS security tooling; and graduate students with a background in deep learning, attention mechanisms, and 3D medical imaging who want a concrete example of ante-hoc interpretability designed into a detection architecture rather than applied after the fact.

Authors’ abstract

The emergence of medical deepfakes, i.e., medical images manipulated by deep generative models, poses a significant threat to clinical workflows. However, existing detectors suffer from two critical limitations: poor generalization to unseen generative architectures for manipulation detection and lack of interpretability. In this context, we present HexMIL (Hierarchical EXplainable Multiple Instance Learning), a mask-free medical deepfake detector that simultaneously addresses both limitations using only binary volume-level supervision. HexMIL decomposes each CT volume into a two-level hierarchy of patches and slices, aggregated via independent Gated Attention modules whose weights are directly combined into a full-resolution 3D attention volume that localizes the manipulated sub-region without any pixel-level annotation. Unlike post-hoc methods such as Grad-CAM, HexMIL's attention weights constitute the exact forward computation driving the classification decision, providing ante-hoc and structurally faithful spatial attribution. We evaluate HexMIL on M3DSynth and CT-GAN datasets under a rigorous cross-generator generalization protocol, training on a single generative architecture and testing on unseen ones. HexMIL outperforms all baselines by $+9.1$ AUC and $+9.4$ F1 in out-of-domain classification, and achieves the best average IoU and Pointing Game score in localization. Project page: opontorno.github.io/hexmil.