LightMIS: Ultra-Lightweight Medical Image Segmentation Without a Stage-Wise Decoder

AI summary

Overview

Research area: Efficient medical image segmentation (computer vision / deep learning for healthcare), specifically ultra-lightweight convolutional architectures for 2D binary segmentation.

Technical level: Advanced. The paper assumes familiarity with U-Net-style encoder-decoder design, depthwise convolutions, GFLOPs/parameter budgeting, the nnU-Net training pipeline, and on-device inference benchmarking.

Scope: The paper introduces LightMIS, a family of three ultra-lightweight convolutional segmentation networks that discard the learned stage-wise decoder in favor of aligning and aggregating the whole five-level encoder hierarchy once, and evaluates them against 15+ representative segmentation models on six public binary segmentation datasets under one shared nnU-Net v2.3.1 protocol.

What This Paper Is About

Most accurate medical image segmentation models are large, and most lightweight ones still rely on a symmetric decoder that reconstructs resolution stage by stage through repeated upsampling and level-by-level skip fusion. LightMIS asks whether that decoder is actually necessary: it takes all five encoder outputs, projects them to a shared channel width, aligns them to one common spatial resolution, aggregates them a single time, and refines the fused result with a compact prediction head. The goal is to hold segmentation accuracy close to much larger models while cutting parameters and compute by one to two orders of magnitude, and to show the resulting networks actually run on a phone GPU.

Key Contributions

1. Progressive Receptive Fusion (PRF) — a lightweight feature-processing module that temporarily expands a feature map's channel width, splits it into three branches with complementary depthwise receptive fields (3×3, dilated 3×3 with rate 2, and 5×5), and passes a channel-averaged spatial response from each branch to the next before convolution.
2. Adaptive Fusion Cascade (AFC) — a sequential composition of the Adaptive Kernel Fusion (AKF) block adapted from AULUNet (with its final ReLU replaced by GELU), the proposed PRF module, and residual adaptive-kernel refinement, applied at every encoder level and once more after aggregation.
3. LightMIS, a scalable decoder-free family — LightMIS-T, LightMIS-S, and LightMIS share identical architecture and differ only in encoder channel widths, using Scale-Aligned Projection (SAP) blocks to align the full encoder hierarchy to a common resolution for compact aggregation followed by AFC refinement.
4. Unified evaluation and on-device benchmarking — integration into nnU-Net and evaluation on DRIVE, Kvasir-SEG, DSB18, BUSI, ISIC-2017, and ISIC-2018 covering overlap accuracy, boundary accuracy, complexity, memory, and latency on desktop hardware and a smartphone GPU.

Main Findings

• Full-model complexity: LightMIS contains 0.131 M parameters and requires 0.575 GFLOPs for a 3×256×256 input.
• Modality-macro accuracy: LightMIS reaches 86.71% macro Dice and 78.99% macro IoU, the second-highest among the compared models after macro-averaging across the five imaging modalities.
• Narrow gap to the best model: Mobile U-ViT obtains 86.75% Dice and 79.07% IoU, so the observed differences are 0.04 and 0.08 percentage points.
• Reductions against larger baselines: Relative to Mobile U-ViT, nnWNet, and nnU-Net, LightMIS cuts parameter count by 90.58–99.61% and GFLOPs by 82.54–96.14%.
• Variant operating points: LightMIS-T uses 0.017 M parameters and 0.099 GFLOPs; LightMIS-S uses 0.038 M parameters and 0.195 GFLOPs; LightMIS uses 0.131 M parameters and 0.575 GFLOPs.
• Per-dataset highlights: LightMIS achieves the highest mean Dice and IoU on DRIVE (82.15% Dice, 69.82% IoU) and the lowest mean ASSD and HD95 on DSB18 (1.53 and 4.59) among the evaluated models. On Kvasir-SEG it obtains 89.87% Dice, 84.25% IoU, 19.14 HD95, and 5.44 ASSD; on BUSI 80.16% Dice, 72.28% IoU, 23.32 HD95, and 8.97 ASSD; on DSB18 91.81% Dice and 85.62% IoU.
• On-device execution: On an Arm Mali-G52 MC2 GPU, all LightMIS variants achieve full GPU delegation, with median delegated latency ranging from 53.31 ms for LightMIS-T to 138.31 ms for LightMIS.
• Data scaling within the family: On Kvasir-SEG, Dice rises from 87.52 (LightMIS-T) to 88.47 (LightMIS-S) to 89.87 (LightMIS); on DRIVE it rises from 81.76 to 81.89 to 82.15.

Methodology in Plain English

The input image is first mapped by a 3×3 convolution to an initial representation. A five-level convolutional encoder then produces a hierarchy of feature maps, with spatial dimensions halved at each transition (H/2^i, W/2^i) via 2×2 average pooling with stride 2 followed by a 1×1 pointwise convolution projecting to the next level's channel width.

Inside each encoder level, the AFC applies three steps in sequence. First, AKF computes two parallel depthwise responses — one with a plain 3×3 kernel and one dilated 3×3 with rate 2 — and mixes them with a per-channel gate computed from global average pooling, using sigmoid activation. Second, PRF doubles the channel count with a 1×1 convolution, splits the result into three uneven groups, and routes it through three different depthwise receptive fields; before each branch is convolved, the channel-wise mean of the previous branch's output is added to it, so local context flows forward across branches. Third, a residual AKF block refines the result, with a learnable scalar γ_A initialized to 0.05 controlling the residual strength.

Instead of decoding, each of the five encoder outputs is passed through a SAP block: a 3×3 depthwise block, a pointwise projection to the shared aggregation width F, and bilinear resizing (align_corners=False) to the spatial size of encoder level 1. The five aligned tensors are concatenated into a 5F-channel tensor, projected back to F channels, and refined by one final AFC. A light prediction head (pointwise convolution plus 1×1 output convolution) produces logits that are bilinearly resized to the input resolution. No symmetric decoding path and no level-by-level encoder–decoder skip fusion are used; only bilinear feature alignment and final logit resizing are retained.

Training follows a shared nnU-Net v2.3.1 2D pipeline: DRIVE images kept at their native 640×640 resolution, all other datasets resized to 256×256, three input channels (grayscale replicated), Lanczos interpolation for images and nearest-neighbor for masks, z-score normalization per channel. All models are trained from scratch for 200 epochs with five-fold cross-validation at the image level, using SGD with Nesterov momentum 0.99, initial learning rate 10⁻², weight decay 3×10⁻⁵, and nnU-Net's polynomial decay schedule. Deep supervision is disabled, no post-processing is applied, and only the final checkpoint is used. The loss is cross-entropy plus soft Dice with equal weighting (ε = 10⁻⁵ for numerical stability), and masks come from pixel-wise argmax over the two softmax probabilities. Metrics are Dice, IoU, HD95, and ASSD, reported as mean ± standard deviation across the five folds, in pixels, at the evaluation resolution with unit pixel spacing.

Datasets: DRIVE (40 images, retinal vessels), BUSI (647 images after excluding 133 normal images; 437 benign and 210 malignant), ISIC-2017 (2750), ISIC-2018 (3694), Kvasir-SEG (1000, polyps), and DSB18 (670, cell nuclei, with instance masks combined by pixel-wise logical union into a single binary mask).

Why This Matters

Research impact: The paper tests whether the stage-wise decoder — a nearly universal component since U-Net — is necessary for competitive medical segmentation once the encoder hierarchy is aligned and fused properly. By matching or approaching far larger models at a fraction of the cost under one standardized nnU-Net protocol, it provides a strong baseline for decoder-free design and a reminder that architecture comparisons in this field are only meaningful under a shared pipeline, which the authors adopt from nnWNet.

Real-world applications:

• Retinal vessel segmentation from fundus photographs, where LightMIS achieves the best mean Dice and IoU among the compared models on DRIVE.
• Polyp detection during colonoscopy, where near-real-time inference matters for clinical workflow.
• Skin lesion analysis from dermoscopy images (ISIC-2017 and ISIC-2018), a common triage setting where smartphone deployment is plausible.
• Breast lesion segmentation from ultrasound and cell nuclei segmentation from microscopy (BUSI and DSB18), where boundary metrics such as ASSD and HD95 matter for measurement.

Industry relevance: The 100% GPU operator coverage and median delegated latency of 53.31–138.31 ms on an Arm Mali-G52 MC2 GPU show that these networks can be fully handed to a mobile GPU rather than partially falling back to the CPU — a practical requirement for point-of-care, mobile, and resource-constrained clinical deployment. The 0.131 M parameter full model and 0.017 M parameter variant also make deployment on very small memory budgets feasible.

Future Directions

• The work is restricted to 2D binary segmentation; whether the same direct-aggregation design transfers to 3D or volumetric inputs is not addressed.
• All six datasets are binary tasks; extending the aggregation head to multi-class segmentation is an open question.
• The latency range of 53.31 ms to 138.31 ms on the Mali-G52 MC2 leaves room for further on-device optimization (for example, additional compression or quantization), which the paper does not explore.
• The comparison set focuses on published architectures trained from scratch; a systematic study of which decoder-free design choices matter most (alignment resolution, aggregation width F, PRF branch layout) would clarify where the gains come from. The ablation studies are referenced in Section 4 but their content is not included in the material available here, and the limitations discussed in Section 5 are likewise not in the provided content.

Target Audience

Researchers and engineers working on efficient or lightweight medical image segmentation, practitioners deploying segmentation models on mobile or embedded clinical hardware, and authors of U-Net variants interested in how much of the decoder is actually needed. Readers looking for per-dataset results on ISIC-2017 and ISIC-2018, ablation tables, memory measurements, or a discussion of limitations should consult the full paper, since those tables and sections are truncated in the content summarized here.

Authors’ abstract

We present LightMIS, a scalable family of ultra-lightweight convolutional networks for 2D binary medical image segmentation without a learned stage-wise decoder. LightMIS aligns the outputs of a five-level encoder to a common resolution using Scale-Aligned Projection blocks, aggregates them once, and refines the fused representation with an Adaptive Fusion Cascade. The cascade combines Adaptive Kernel Fusion with the proposed Progressive Receptive Fusion module, which uses temporary channel expansion, complementary depthwise receptive fields, and progressive cross-branch information transfer. We evaluate LightMIS-T, LightMIS-S, and LightMIS using five-fold cross-validation under a common nnU-Net v2.3.1 protocol on DRIVE, Kvasir-SEG, DSB18, BUSI, ISIC-2017, and ISIC-2018. Full LightMIS contains 0.131 M parameters and requires 0.575 GFLOPs for a $3\times256\times256$ input, achieving modality-macro Dice and IoU scores of 86.71% and 78.99%, respectively. Mobile U-ViT obtains 86.75% Dice and 79.07% IoU, so the observed differences are 0.04 and 0.08 percentage points. Relative to Mobile U-ViT, nnWNet, and nnU-Net, LightMIS reduces parameter count by 90.58$-$99.61% and GFLOPs by 82.54$-$96.14%. On an Arm Mali-G52 MC2 GPU, all LightMIS variants achieve full GPU delegation, with median delegated latency ranging from 53.31 ms for LightMIS-T to 138.31 ms for LightMIS. These results demonstrate a favorable accuracy$-$complexity trade-off and on-device execution feasibility for the evaluated tasks. The code is publicly available at https://github.com/AndreiiArhire/LightMIS.