Vascular anatomy-aware self-supervised pre-training for X-ray angiogram analysis

AI summary

Overview

Research area: Medical image analysis, specifically self-supervised pre-training (SSL) for X-ray coronary angiography, at the intersection of computer vision and interventional cardiology.

Technical level: Advanced. The paper assumes familiarity with masked image modeling, Vision Transformers, contrastive learning, and clinical segmentation/detection pipelines.

Scope: The paper introduces VasoMIM, a vascular-anatomy-aware masked image modeling framework, together with XA-170K, and evaluates it on four downstream tasks across six datasets.

What This Paper Is About

Deep learning for X-ray angiogram analysis is bottlenecked by a shortage of pixel-level annotated data, and annotating vessels and stenoses requires specialized clinical expertise. Self-supervised learning could help, but the authors argue two things are missing in this domain: a domain-specific SSL framework and a large-scale pre-training dataset. This paper supplies both, injecting anatomical knowledge about vessel structure directly into the pre-training objective and scaling pre-training data from 20K images in the earlier conference version to 170K.

Key Contributions

1. VasoMIM framework. A self-supervised pre-training framework for X-ray angiogram analysis that integrates vascular anatomical knowledge, built on two designs: an anatomy-guided masking strategy and an anatomical consistency loss.
2. XA-170K dataset. Described as the largest publicly available dataset for X-ray angiogram pre-training, assembled from four public sources (CADICA, SYNTAX, XCAD, CoronaryDominance) for a total of 171,478 images. The authors state this provides the first empirical verification of scaling laws for MIM in this specific domain (the scaling-law analysis itself falls in the truncated portion of the paper).
3. Broad empirical validation. Experiments span four downstream tasks (vessel segmentation, vessel segment segmentation, stenosis segmentation, stenosis detection) across six datasets, with five random seeds per configuration, showing gains over state-of-the-art SSL alternatives.
4. Extension over the AAAI 2026 preliminary version. Relative to VasoMIM-v1, the extended version adds data scaling (20K to 170K), integrates the segmentor's probability map as co-guidance alongside the Frangi filter to reduce noise from bony structures, expands validation from one task to four clinical applications, and adds detailed design-choice analysis.

Main Findings

• Segmentation state of the art. VasoMIM achieves the best average rank (1.00 ± 0.00) across the segmentation benchmarks in Table I. On ARCADE-V it reaches 80.25 ± 0.12 DSC and 82.06 ± 0.18 clDice; on CAXF 89.68 ± 0.08 DSC and 92.57 ± 0.12 clDice; on XCAV 86.09 ± 0.09 DSC and 84.12 ± 0.17 clDice; on ARCADE-S 55.62 ± 0.63 DSC; on ARCADE-VS 58.87 ± 0.15 DSC.
• Large gains over training from scratch. Compared with UNet trained from scratch, VasoMIM improves DSC by 8.81 (ARCADE-V), 6.92 (CAXF), 7.91 (XCAV), 28.58 (ARCADE-S), and 20.10 (ARCADE-VS), with clDice gains of 11.39, 7.62, and 9.41 respectively.
• Pre-training beats architectural sophistication. A pre-trained UNet outperforms advanced segmentors trained from scratch (TransUNet, VM-UNet, RWKV-UNet), which the authors read as evidence that large-scale unlabeled pre-training matters more than decoder complexity in this domain.
• Domain-specific pre-training outperforms a natural-image foundation model. DINOv3, pre-trained on LVD-1698M (1.6 billion images), lags behind VasoMIM on the segmentation benchmarks, despite being the leading vision foundation model in the comparison.
• Stenosis detection gains. On the Stenosis detection dataset, VasoMIM reaches 94.91 ± 0.28 mAP50, 25.72 ± 1.57 mAP75, and 41.07 ± 0.35 mAP, improving on Faster R-CNN trained from scratch by 6.54, 6.71, and 4.44 points respectively. Against the strongest SSL competitor, LocalMIM, it adds +1.32% mAP50, +0.96% mAP75, and +0.83% mAP.
• Both components contribute, and combine. In the ablation on ARCADE-V and XCAV, starting from vanilla MAE (γ = 0.5), adding only the anatomical consistency loss raises ARCADE-V DSC by 0.54 and clDice by 0.54; adding only anatomy-guided masking raises them by 0.56 and 0.36; using both raises them by 0.94 and 1.01. On XCAV the respective gains are 1.27/1.41, 1.40/1.53, and 1.57/1.87.
• Advantage widens on harder tasks. The margin over baselines is largest on stenosis segmentation (ARCADE-S) and vessel segment segmentation (ARCADE-VS), which the authors attribute to better capture of fine-grained topological detail.
• Statistically significant improvement over the conference version. VasoMIM surpasses VasoMIM-v1 across all segmentation and detection metrics (p-value = 1.18 × 10⁻⁴, paired t-test).
• Frangi filter alone is weak. As a traditional baseline, the Frangi filter scores 41.30 DSC / 40.91 clDice on ARCADE-V, 64.01 / 65.73 on CAXF, and 58.46 / 57.15 on XCAV — far below learned methods.

Methodology in Plain English

The approach starts from the observation that vessels occupy a tiny fraction of an angiogram, so standard masking strategies mostly hide background and teach the model little about vasculature.

1. Extract anatomy without labels. A Hessian-based Frangi filter computes a vesselness response across four scales (σ = 1, 2, 3, 4), thresholds it at the 92nd percentile, and applies region growing from the global maximum to produce a binary vascular mask B.
2. Add a learned, smoother anatomy map. A lightweight UNeXt-S segmentor (0.26 M parameters) is trained on the Frangi outputs to produce a probability map M. The two are blended into a co-guidance map G = η·B + (1 − η)·M with η = 0.5, which reduces sensitivity to noise and to bony structures.
3. Mask where the vessels are. The image and guidance map are split into patches, and each patch receives a sampling weight proportional to its vessel content. A weak-to-strong schedule controls how much guidance is used: β_e ramps linearly from β_0 to β_E across epochs, so β_e·γ·N patches are masked anatomically and (1 − β_e)·γ·N randomly, where γ is the masking ratio.
4. Reconstruct with anatomical supervision. Beyond the standard pixel-level reconstruction loss (MSE), an anatomical consistency loss compares the vessel segmentation of the original image against that of the reconstruction, using the differentiable UNeXt-S segmentor as the semantic extractor and cross-entropy as the metric. The total objective is L_MIM = L_rec. + L_cons.
5. Pre-train at scale, then fine-tune. Pre-training uses ViT-B/16 for 800 epochs, taking roughly ten hours on 8 × NVIDIA H20 GPUs. Downstream segmentation uses UNet at 224 × 224 with a SimpleFPN adaptor and AdamW with cosine annealing; detection uses Faster R-CNN with FPN at 512 × 512 for 100 epochs with ViTDet configurations.

Why This Matters

Impact on research. The paper argues that anatomy-aware inductive bias, not just scale, is what unlocks SSL for angiograms, and it offers the domain's first dedicated MIM framework and largest pre-training dataset. It also shows that generic natural-image and general medical foundation models transfer imperfectly to this modality, reinforcing the case for modality-specific pre-training.

Real-world applications:

• Automated vessel segmentation to help clinicians delineate vascular topology in low-contrast, motion-artifact-heavy angiograms.
• Stenosis segmentation and detection to support identification of narrowing in coronary arteries.
• Vessel segment segmentation to support anatomical labeling and intervention planning.
• Reducing the annotation burden: the framework learns from unlabeled angiograms, which are abundant, rather than requiring pixel-level labels that are labor-intensive and dependent on specialist knowledge.

Industry relevance. Co-authors are affiliated with the DAMO Academy at Alibaba Group and Hupan Lab, alongside the Chinese Academy of Sciences and the University of Chinese Academy of Sciences. The authors state that VasoMIM and XA-170K will be released at https://github.com/Dxhuang-CASIA/XA-SSL, which would make the framework and pre-training corpus directly reusable by other groups and by commercial angiography software developers.

Future Directions

• Full scaling-law characterization. The authors claim the first empirical verification of scaling laws for MIM in this domain and state they systematically study both data scale and model capacity, but the detailed scaling results are not present in the available text.
• Scaling beyond ViT-B/16. The default backbone is ViT-B/16; whether larger capacities continue to help, and at what compute cost, is a natural open question given the stated model-capacity scaling analysis.
• Closing the gap on stenosis segmentation. ARCADE-S remains the weakest absolute result (55.62 DSC), suggesting lesion-level segmentation is still substantially harder than vessel-level tasks.
• Broadening task coverage. Validation covers four tasks on six datasets; extending the foundation model to other angiographic procedures, additional anatomical territories, or multimodal clinical data is not explored in what is reported here.

Target Audience

Researchers and engineers working on medical image analysis, self-supervised learning, and vision foundation models, particularly those focused on interventional cardiology and X-ray angiography. It is also relevant to clinical translation teams evaluating whether pre-trained models can reduce annotation cost for vessel and stenosis analysis. Readers without a background in masked image modeling or transformer-based vision will find the method sections demanding.

Authors’ abstract

X-ray angiography is the gold standard imaging modality for cardiovascular diseases. However, current deep learning approaches for X-ray angiogram analysis are severely constrained by the scarcity of annotated data. While large-scale self-supervised learning (SSL) has emerged as a promising solution, its potential in this domain remains largely unexplored, primarily due to the lack of effective SSL frameworks and large-scale datasets. To bridge this gap, we introduce a vascular anatomy-aware masked image modeling (VasoMIM) framework that explicitly integrates domain-specific anatomical knowledge. Specifically, VasoMIM comprises two key designs: an anatomy-guided masking strategy and an anatomical consistency loss. The former strategically masks vessel-containing patches to compel the model to learn robust vascular semantics, while the latter preserves structural consistency of vessels between original and reconstructed images, enhancing the discriminability of the learned representations. In conjunction with VasoMIM, we curate XA-170K, the largest X-ray angiogram pre-training dataset to date. We validate VasoMIM on four downstream tasks across six datasets, where it demonstrates superior transferability and achieves state-of-the-art performance compared to existing methods. These findings highlight the significant potential of VasoMIM as a foundation model for advancing a wide range of X-ray angiogram analysis tasks. VasoMIM and XA-170K will be available at https://github.com/Dxhuang-CASIA/XA-SSL.