Adaptive Morph-Patch Transformer for Aortic Vessel Segmentation

AI summary

Overview

• Research area: Medical image segmentation (computer vision applied to cardiovascular imaging), specifically 3D/2D aortic vessel segmentation from CT angiography.
• Technical level: Advanced. The paper assumes familiarity with Vision Transformers, deformable convolution/deformable patch embedding, diffeomorphic deformation fields, Swin-style window attention, and nnU-Net training pipelines.
• Scope: The paper proposes an adaptive patch-partitioning Transformer (MPT) that shapes image patches around vascular morphology and clusters semantically similar patches, then benchmarks it against 2D, 3D, hybrid ViT-CNN, and Mamba-based segmentation baselines on three open aortic datasets.

What This Paper Is About

Transformer-based segmentation models split medical volumes into fixed-size rectangular patches, which cuts across thin, branching, irregularly shaped blood vessels and damages their structural continuity. This paper builds an alternative in which patch shapes are generated adaptively from the image itself, so that each patch follows the vessel's morphology, and adds an attention mechanism that groups patches with similar semantics regardless of where they sit. The goal is more accurate and topologically faithful segmentation of the aorta and its branches, including fine vessels and diseased structures such as aortic dissection.

Key Contributions

1. Adaptive Morph-Patch Transformer (MPT): a patch-partitioning strategy in which a CNN predicts a stationary velocity field that is integrated (via the scaling-and-squaring method) into a diffeomorphic deformation field. Patches are then sampled at the deformed coordinates, so patch boundaries follow vascular structures and preserve topological continuity, unlike the rectangular, directly-learned deformation fields of deformable patch embedding (DPT) or deformable convolutions.
2. Semantic Clustering Attention (SCA): a soft K-means module that extracts core semantic features, updates them differentiably using a smoothed cluster-assignment function, and uses the updated centers as keys/values in an attention operation, so features from semantically similar patches are aggregated regardless of spatial position.
3. A full architecture and three model variants: MPT (pure 3D ViT-based), MPT-UNETR (hybrid 3D ViT-CNN), and MPT-UNet (lightweight 2D), all embedded in a 3D UNet-like framework that combines Swin-style window/shifted-window attention with SCA.
4. Validation on three open datasets (AVT, AortaSeg24, TBAD) with state-of-the-art claims, plus an ablation isolating each component and a transfer test applying the Morph-Patch strategy inside a Mamba backbone.

Main Findings

• AVT (56 cases, single-class aorta): MPT reaches Dice 0.856 (SD 0.106), mIoU 0.762 (SD 0.150), clDice 0.757 (SD 0.152). MPTUNet reaches Dice 0.896 (SD 0.046), mIoU 0.815 (SD 0.073), clDice 0.839 (SD 0.078). For comparison, DPT scores Dice 0.886 (SD 0.055), mIoU 0.800 (SD 0.086), clDice 0.825 (SD 0.089), and MambaVision scores Dice 0.882 (SD 0.064), mIoU 0.795 (SD 0.097), clDice 0.795 (SD 0.099).
• TBAD (100 CTA scans of Type-B Aortic Dissection, true lumen / false lumen / false lumen thrombus): MPT scores Dice 0.933 (SD 0.150), mIoU 0.881 (SD 0.161), clDice 0.915 (SD 0.032); MPTUNet scores Dice 0.930 (SD 0.147), mIoU 0.877 (SD 0.156), clDice 0.920 (SD 0.031).
• AortaSeg24 (100 CTA scans, 23 annotated regions): MPTUNETR achieves the best Dice (0.809) and mIoU (0.695) among all compared methods; MPT scores Dice 0.804 (SD 0.021), mIoU 0.690 (SD 0.024), clDice 0.926 (SD 0.032); MPTUNet scores Dice 0.796 (SD 0.075), mIoU 0.686 (SD 0.077), clDice 0.966 (SD 0.011), edging DSCViT (Dice 0.788, clDice 0.965) and MambaVision (Dice 0.795, clDice 0.960).
• Topology preservation: MPTUNet records the highest clDice across all three datasets — 0.839 (AVT), 0.966 (AortaSeg24), and 0.920 (TBAD) — which the authors attribute to the morph-patch strategy maintaining continuity of the aortic structure.
• Ablation on AVT: a plain ViT backbone scores Dice 0.834, mIoU 0.743, clDice 0.732. Adding the Morph-Patch strategy gives Dice 0.839, mIoU 0.748, clDice 0.744. Adding SCA on top gives Dice 0.856, mIoU 0.762, clDice 0.757. The clDice gain from Morph-Patch (0.732 to 0.744) is highlighted as evidence of better topology preservation.
• Generalization to a Mamba backbone: substituting a Mamba architecture, the Morph-Patch strategy raises clDice from 0.711 to 0.725 (Dice moves from 0.829 to 0.831, mIoU stays at 0.730).
• Efficiency: the authors report that MPT-based models achieve superior segmentation while using significantly fewer parameters and lower computational cost than many recent state-of-the-art methods; the specific FLOPs and parameter counts are presented only as plots in Figure 3 and are not given as numbers in the text.
• Interpretability of the deformation field: inspection of nine representative deformation fields from the three datasets shows deformations concentrate around the spine and major vascular structures. The authors interpret this as the model implicitly using the stable anatomical relationship between the vertebral column and major vessels as a prior for localizing vascular structures.
• Visual results: on TBAD, only nnFormer, UNETR, SwinUNETR, and the MPT-based methods were able to display all three anatomical structures, with the MPT variants producing more complete true-lumen segmentations. On AortaSeg24, MPT-based methods reduce incomplete anatomical structures, noise, and misclassifications.

Methodology in Plain English

The framework starts from a 3D UNet-like design. In each stage, instead of cutting the feature map into fixed rectangles:

1. Predict where patches should be. A small CNN reads the features and outputs a velocity field, a vector at every location describing how points should move.
2. Turn velocities into a smooth, invertible warp. The velocity field is integrated over time from 0 to 1 using the scaling-and-squaring method, producing a deformation field (a set of coordinate offsets). Because the transformation comes from integrating a velocity field, it is diffeomorphic — smooth, invertible, and unable to tear or fold the image, which preserves the continuity of vessels.
3. Sample morphology-aware patches. Features are resampled with bilinear interpolation at the displaced coordinates, so each patch adapts its shape to the vessel rather than the vessel being cut by a fixed grid.
4. Attend within and across patches. A Swin-style window and shifted-window attention captures spatial relationships inside and between neighboring patches.
5. Group patches by meaning, not location. Semantic Clustering Attention runs a differentiable soft K-means: it maintains a set of core semantic feature vectors, computes soft (exponential) membership weights instead of hard cluster assignments, updates the centers, and then uses those updated centers as the keys and values in an attention operation. This lets patches that look semantically alike exchange information even if they are far apart in the image, which helps with vessels of very different sizes.

Training follows nnU-Net conventions: PyTorch on NVIDIA GeForce RTX 3090 GPUs on Ubuntu 20.04, official baseline code reproduced inside nnU-Net, nnU-Net normalization/resampling/cropping, an 8:1:1 train/validation/test split, images standardized to 128×128×128 for 3D and 512×512 for 2D, Adam optimizer with learning rate 5×10⁻⁵, Dice loss, 1000 epochs, and 32 clusters in the attention module.

Why This Matters

Impact on research: The paper challenges a default assumption in medical Vision Transformers — that patches must be axis-aligned rectangles. It shows that a diffeomorphic, velocity-field-driven partition can be dropped into a standard architecture and that a piece of it (the morph-patch strategy) transfers to a different backbone family (Mamba), suggesting the idea is architecture-agnostic rather than a one-off design. The use of topology-aware clDice alongside Dice and mIoU also reinforces that overlap metrics alone can hide broken vessel trees.

Real-world applications:

• Diagnosing and characterizing aortic aneurysms, dissections, and stenosis, where accurate delineation of the lumen and thrombus is clinically decisive.
• Surgical and endovascular procedure planning that requires a clean 3D model of the aorta and its branches.
• Computational flow modeling (CFD) of blood through the aorta, which depends on a watertight, topologically correct vessel surface.
• Monitoring disease progression over serial scans, where small changes in vessel caliber or thrombus burden must be measured reliably.

Industry relevance: Medical imaging software vendors, hospital radiology AI pipelines, and surgical navigation/robotics platforms all depend on segmentation backbones that are both accurate and cheap enough to deploy. The paper's emphasis on competitive accuracy with fewer parameters and lower computation (relative to several recent state-of-the-art baselines) is directly relevant to deployment constraints, and the release of code at https://github.com/iCherishxixixi/MPTransformer supports reproducibility and integration.

Future Directions

• Broader validation: the authors state they intend to evaluate MPT on a wider range of datasets and anatomical scenarios beyond the three aortic datasets used here, with the aim of clinical adoption.
• Deeper use of anatomical priors: the deformation fields consistently cluster around the spine and major vessels. Whether this spine–aorta relationship can be modeled explicitly, or exploited for other thoracoabdominal targets, is left open.
• Extending the morph-patch idea beyond vessels: the method is motivated by tubular, branching structures; whether it helps for other non-rectangular or thin anatomical targets (nerves, airways, small lesions) is untested here.
• Component interaction and tuning: the number of semantic clusters was fixed at 32 and training ran for 1000 epochs; the paper does not report sensitivity analyses for these settings, leaving room to study how cluster count, velocity-field regularization, and patch deformation magnitude affect performance and stability.

Target Audience

Researchers and graduate students working on medical image segmentation and Vision Transformer architecture design; applied scientists at medical imaging or surgical-navigation companies building aorta segmentation pipelines; and clinical researchers in cardiovascular imaging who want a plain-language view of what modern adaptive-patch segmentation can and cannot yet deliver. Readers need prior familiarity with Transformer attention and nnU-Net-style evaluation to follow the method and tables fully.

Authors’ abstract

Accurate segmentation of aortic vascular structures is critical for diagnosing and treating cardiovascular diseases.Traditional Transformer-based models have shown promise in this domain by capturing long-range dependencies between vascular features. However, their reliance on fixed-size rectangular patches often influences the integrity of complex vascular structures, leading to suboptimal segmentation accuracy. To address this challenge, we propose the adaptive Morph Patch Transformer (MPT), a novel architecture specifically designed for aortic vascular segmentation. Specifically, MPT introduces an adaptive patch partitioning strategy that dynamically generates morphology-aware patches aligned with complex vascular structures. This strategy can preserve semantic integrity of complex vascular structures within individual patches. Moreover, a Semantic Clustering Attention (SCA) method is proposed to dynamically aggregate features from various patches with similar semantic characteristics. This method enhances the model's capability to segment vessels of varying sizes, preserving the integrity of vascular structures. Extensive experiments on three open-source dataset(AVT, AortaSeg24 and TBAD) demonstrate that MPT achieves state-of-the-art performance, with improvements in segmenting intricate vascular structures.