ProPL: Universal Semi-Supervised Ultrasound Image Segmentation via Prompt-Guided Pseudo-Labeling

AI summary

Overview

• Research area: Medical computer vision, specifically ultrasound image segmentation, semi-supervised learning, and "universal" (multi-task, multi-organ) segmentation models.
• Technical level: Advanced. The paper assumes familiarity with semi-supervised learning paradigms (pseudo-labeling, consistency regularization), transformer-style attention blocks, prompt/encoder-decoder architectures, and standard segmentation metrics.
• One-sentence scope: The paper defines and benchmarks the new task of universal semi-supervised ultrasound image segmentation, introduces the ProPL framework to address it, and releases a 6,400-image, 5-organ, 8-task ultrasound dataset for evaluation.

What This Paper Is About

Ultrasound segmentation models in the literature are almost always built and trained for one specific organ or one specific task, so a clinic needing to segment several structures has to maintain several separate models — and most of those models need large amounts of expert-annotated data, which is expensive and hard to obtain for noisy ultrasound images. This paper asks whether a single model can handle many ultrasound segmentation tasks at once while training mostly on unlabeled images. The authors propose ProPL, which shares one visual encoder across tasks, uses text prompts to tell the model which task it is doing, and refines automatically generated pseudo-labels for unlabeled data using prediction uncertainty.

Key Contributions

1. A new problem setting. The authors pioneer the investigation of universal semi-supervised ultrasound image segmentation, combining multi-task/multi-organ coverage with learning from both labeled and unlabeled data — a combination they state has not previously been studied for ultrasound.
2. The ProPL framework. ProPL uses a shared vision encoder (ConvNeXt-Tiny), a prompt encoder (BERT) that turns a text instruction into task-specific features, and two independent decoders. Two mechanisms are central: prompting-upon-decoding, which injects task prompts into the decoders via cross-attention, and uncertainty-driven pseudo-label calibration (UPLC), which rescales pseudo-labels according to their measured variance.
3. A released benchmark dataset. A compiled ultrasound dataset of 6,400 images spanning 5 organs and 8 distinct segmentation tasks, with 800 samples per task, aggregated from public sources (Dataset B, BUSI, HC18, CAMUS, HMC-QU, MMOTU, TN3K, DDTI) using a 3:1 train-test split.
4. Extensive comparative evaluation. Experiments under 1/16, 1/8, and 1/4 labeled-data partitions against single-task supervised, single-task semi-supervised, universal supervised, and universal semi-supervised baselines, plus ablations isolating each component.

Main Findings

• Best overall accuracy under the 1/16 partition: ProPL reaches 81.13 mean Dice and 71.96 mIoU, exceeding the best single-task supervised model by 5.18% mean Dice and 5.87% mIoU, the top single-task semi-supervised model by 0.97%, the leading universal supervised model by 17.36% and 20.59%, and Univ-full by 33.74% and 35.98%.
• Consistent gains at higher label budgets: Under the 1/8 partition ProPL reaches 83.35 mDice and 74.83 mIoU, outperforming the second-best model by 2.2% in mean Dice. Under the 1/4 partition it reaches 85.50 mDice and 77.46 mIoU, improvements of 1.32% mean Dice and 1.82% mean IoU over the second-best model.
• Boundary accuracy: On HD95 (lower is better), the method attains 20.59, outperforming the second-best result by 1.06 under the 1/8 setting.
• Task prompts are essential: Removing task prompts causes drops of 19.59% mean Dice and 18.06% mIoU under the 1/16 partition, with similar degradation observed at 1/8 and 1/4.
• Prompting-upon-decoding beats prior prompting schemes: The approach outperforms the prompting methods implemented in CLIP-UM and DoDNet across all eight tasks, with gains of 3.05% and 1.16% mean Dice respectively.
• UPLC improves pseudo-label quality: Removing pseudo-label calibration reduces performance by 2.5% in average Dice and 3.4% in IoU under 1/16, with consistent degradation at 1/8 and 1/4.
• Two perturbations is the practical sweet spot: Optimal performance occurs with two stochastic perturbations under the 1/4 and 1/8 protocols (85.61% and 83.43%) and three perturbations under 1/16 (81.13%). GPU memory constraints prevented testing beyond four perturbations. Each additional perturbation costs 2.43 GB of memory and 27.6 s per epoch.
• Dropout beats Gaussian noise as the perturbation strategy: Gaussian noise performs worse, with a 2.58 mIoU drop.
• Efficient memory footprint: ProPL achieves the highest Dice among compared methods while using 712 MB of memory.
• Prompting cost is small: Compared to one-hot prompting, the text prompting adds only 18 s/epoch.
• Generalization to unseen data: On an in-house external set of 700 images, the model achieves 73.33 mIoU, comparable to internal results.
• One notable exclusion: VerSemi (Zeng et al. 2025), a pioneering universal semi-supervised method for CT/MRI, was not included in comparisons because in the authors' experiments it performed mediocrely on ultrasound images.
• Not reported: The paper does not report inference latency, parameter counts, or per-task standard deviations; only the metrics listed above (Dice, IoU/mIoU, HD95, memory, and training time) appear.

Methodology in Plain English

The framework has four pieces: a shared vision encoder, a prompt encoder, and two decoders that share the same encoder features.

Encoding. Each ultrasound image passes through a ConvNeXt-Tiny backbone that produces multi-scale features across four stages. Separately, a short text instruction such as "Segment the breast cancer in the ultrasound image" passes through a BERT encoder to become a task embedding. The target word can be swapped for other structures such as the left atrium, ovarian tumor, or thyroid gland, so one model covers all tasks by changing the sentence.

Prompting-upon-decoding. At each decoding stage, the visual features are first refined with multi-head self-attention. The text embedding is then passed through a 1D convolution and a linear projection to match dimensions, and multi-head cross-attention lets the visual features query the prompt. A learnable scalar controls how strongly the prompt influences the output. Skip connections follow UNETR, and sub-pixel convolutions perform upsampling.

Training on labeled data. For labeled images, one decoder produces a mask trained against ground truth using a combination of binary cross-entropy and Dice losses.

Training on unlabeled data. For unlabeled images, the first decoder produces a pseudo-label and the second decoder predicts a mask from the same features; the two are forced to agree. The key twist is that the pseudo-label is not trusted blindly. The encoded visual representation is perturbed N times (via dropout at rate 0.3), producing an ensemble of pseudo-masks. The mean of these masks is treated as the label, and their variance is treated as an uncertainty map. The final rectified pseudo-label multiplies the mean by exp(−variance), so regions where the model is unstable contribute a weaker training signal.

Training setup. Images are resized to 224×224 and augmented with random rotation and scaling. Training uses SGD with momentum 0.9 and weight decay 1e-5, batch size 16, an initial learning rate of 0.001 with a polynomial schedule (power 0.9), and 200 epochs on a single NVIDIA GeForce RTX 4090. Experiments use 1/16, 1/8, and 1/4 of training data as labeled, with the rest unlabeled.

Why This Matters

Impact on research. The paper opens a task — universal semi-supervised ultrasound segmentation — that had no prior dedicated treatment, and pairs it with a public benchmark dataset. It also shows that combining task prompts with uncertainty-aware pseudo-labeling handles the notorious difficulty of ultrasound (speckle noise, acoustic shadows, tissue-dependent artifacts that blur anatomical boundaries). The comparison across four categories of baselines gives the community a clear reference point, and the finding that a strong CT/MRI method underperforms on ultrasound signals that modality-specific work is still needed.

Real-world applications:

• Point-of-care and bedside ultrasound, where a single deployed model could serve cardiology (left atrium, left ventricle, myocardium), obstetrics (fetal head), breast cancer screening, ovarian tumor assessment, and thyroid gland/nodule evaluation.
• Screening programs in settings with few expert sonographers, where most images can remain unlabeled and only a small annotated fraction is needed.
• Computer-aided diagnosis and surgical planning pipelines that currently require separate organ-specific models to be trained and maintained.
• Rapid adaptation to a new organ or task at a new hospital by editing the text prompt rather than retraining a bespoke network.

Industry relevance. The modest 712 MB memory footprint, the small 18 s/epoch prompting overhead, and the accuracy at the 1/16 label budget all point toward deployment feasibility on affordable hardware. The involvement of MedAI Technology (Wuxi) and the released code at https://github.com/WUTCM-Lab/ProPL suggest direct translational intent. The annotation-cost reduction is the commercial lever: a hospital can reach useful accuracy from a fraction of the labeled data it would otherwise need.

Future Directions

• Extending to other modalities and organs. The authors explicitly note that VerSemi, the closest universal semi-supervised work, was designed for CT/MRI and performed mediocrely on ultrasound, so a single framework spanning ultrasound plus CT/MRI remains open.
• Scaling the perturbation ensemble. UPLC's accuracy improves with more perturbations, but GPU memory constraints prevented testing beyond four; finding cheaper ways to estimate uncertainty — or applying the Gaussian-noise variant more cleverly, given its 2.58 mIoU deficit — is an obvious follow-up.
• Better task-prompt representations. Text prompts are shown to be essential (removing them collapses performance by 19.59% mean Dice at 1/16), yet the paper uses minimal templated sentences. Richer or clinically natural language prompts, and comparison against one-hot prompting, are natural extensions.
• Rigorous external validation at scale. Generalization was demonstrated on a small in-house set of 700 images reaching 73.33 mIoU; larger, multi-center, prospectively collected evaluations would be needed to confirm clinical robustness.

Target Audience

Researchers and graduate students in medical image analysis and semi-supervised learning; engineers building clinical ultrasound decision-support or segmentation products; clinicians and sonographers interested in how much annotation a usable multi-organ model actually requires; and benchmark builders looking for a ready-made multi-task ultrasound dataset. Beginners will grasp the motivation and results but will need background in semi-supervised segmentation and attention mechanisms to follow the methodology in full.

Authors’ abstract

Existing approaches for the problem of ultrasound image segmentation, whether supervised or semi-supervised, are typically specialized for specific anatomical structures or tasks, limiting their practical utility in clinical settings. In this paper, we pioneer the task of universal semi-supervised ultrasound image segmentation and propose ProPL, a framework that can handle multiple organs and segmentation tasks while leveraging both labeled and unlabeled data. At its core, ProPL employs a shared vision encoder coupled with prompt-guided dual decoders, enabling flexible task adaptation through a prompting-upon-decoding mechanism and reliable self-training via an uncertainty-driven pseudo-label calibration (UPLC) module. To facilitate research in this direction, we introduce a comprehensive ultrasound dataset spanning 5 organs and 8 segmentation tasks. Extensive experiments demonstrate that ProPL outperforms state-of-the-art methods across various metrics, establishing a new benchmark for universal ultrasound image segmentation.