A Systematic Benchmark of Intensity Normalisation Methods for 3D Knee MRI Segmentation and Cross-Domain Generalisability

AI summary

Overview

• Research area: Medical image analysis, specifically deep learning for 3D knee MRI segmentation, preprocessing (intensity normalisation) pipelines, and out-of-distribution generalisability of segmentation models.
• Technical level: Intermediate. The clinical motivation and the headline result are accessible to a general reader, but the evaluation design (linear mixed-effects modelling, cross-validation fold-level variance decomposition) assumes some familiarity with machine learning benchmarking.
• One-sentence scope: The paper benchmarks seven intensity normalisation methods on an otherwise identical 3D U-Net meniscus segmentation pipeline, trained on IWOAI 2019 and tested both internally and on the external SKM-TEA dataset, to isolate how much normalisation choice matters relative to domain shift.

What This Paper Is About

MRI intensities have no fixed physical meaning, so the same tissue can appear at very different brightness values across scanners, protocols, and patients. This makes models trained at one centre unreliable when deployed at another. The paper asks a narrow, practical question: of the standard, no-retraining-required intensity normalisation methods available, does the choice of method measurably change segmentation performance, and does it help when the test data comes from a different institution and imaging protocol? The authors use meniscus segmentation from knee MRI as the test case because the meniscus is small, irregular, and low-contrast — conditions that should expose any preprocessing advantage.

Key Contributions

1. A controlled head-to-head benchmark of seven normalisation methods — image-wise Z-score, min-max scaling, robust min-max scaling (1st–99th percentile clipping), global histogram equalisation (HE), contrast-limited adaptive histogram equalisation (CLAHE), Nyúl histogram matching, and a Gaussian Mixture Model (GMM)-based method adapted from white stripe normalisation — all trained with identical 5-fold cross-validation splits, batch size, patch size, and 500-epoch schedules.
2. External validation in musculoskeletal MRI, a setting the authors identify as underexplored: prior systematic work on normalisation and domain shift concentrated on brain (and breast) MRI, and the paper notes there is no knee-MRI equivalent of white stripe normalisation.
3. A statistical framework that separates method effects from image and fold variability, using linear mixed-effects models with random intercepts for fold and test image, Type III ANOVA with Satterthwaite's method, and Tukey-adjusted pairwise comparisons, applied to both volumetric (Dice, CCC, MAE) and distance (HD95) metrics.
4. A quantified comparison of effect sizes: normalisation method explained only 0.5% of total variance in external Dice scores, while the internal-to-external performance drop across all methods was roughly 10% Dice, framing normalisation as a marginal rather than a decisive lever.

Main Findings

• Internal test performance was nearly flat across methods. On the IWOAI 2019 test split, min-max (89.05 ± 0.07% DSC) and GMM (89.05 ± 0.14% DSC) had the highest mean Dice, with Z-score close behind (89.04 ± 0.12%). HE was the weakest method across all metrics (88.68 ± 0.09% DSC). A Type III ANOVA found a significant overall effect of method (F(6,942) = 8.08, p < 0.001), but pairwise comparisons found few significant differences: the top three methods significantly outperformed HE and CLAHE, and HE performed significantly worse than all methods apart from CLAHE.
• No significant internal effect on boundary distance. HD95 showed no significant effect of method on internal data (F(6,942) = 1.71, p = 0.12).
• External performance dropped sharply for every method. On SKM-TEA, Dice fell from the high-80s to roughly 78–79% across the board. Nyúl had the highest external Dice (79.10 ± 0.34%), followed by Z-score (78.95 ± 0.28%) and CLAHE (78.86 ± 0.26%); HE was lowest (78.31 ± 0.24%), with min-max (78.45 ± 0.23%), robust min-max (78.48 ± 0.26%), and GMM (78.44 ± 0.14%) between them.
• External differences were small but statistically significant. The mixed-effects model found a significant method effect (F(6,5260) = 79.06, p < 0.001). Relative to Z-score (reference, estimated DSC = 78.95%, 95% CI [78.28, 79.63]), the estimated Dice differences were −0.51 for min-max, −0.48 for robust min-max, −0.65 for HE, −0.09 for CLAHE, +0.14 for Nyúl, and −0.51 for GMM. Nyúl, Z-score, and CLAHE all significantly outperformed min-max, robust min-max, HE, and GMM (p < 0.001). Nyúl beat CLAHE significantly (p < 0.001) and had the best volumetric CCC and MAE, but its advantage over Z-score was not statistically significant (p = 0.053) even though the fixed-effect confidence interval excluded zero.
• Almost all external variance came from the images, not the method. The intra-class correlation coefficient for test image was 0.945, versus 0.003 for cross-validation fold, and the marginal R² was 0.005 — meaning normalisation method explained only 0.5% of total variance.
• No significant external effect on HD95 either (F(6,5264) = 1.29, p = 0.26), despite CLAHE achieving the lowest mean HD95 (3.05 ± 0.23 mm). The paper reads this as evidence that segmentation shape was consistent across methods.
• Method-specific behaviour differed between domains. Min-max suffered the largest internal-to-external drop, which the authors attribute to sensitivity to dataset-specific intensity outliers. Nyúl showed the smallest drop. CLAHE, which underperformed internally, ranked among the top methods externally. The GMM method performed well internally but less well externally, which the authors link to differences in intensity profiles between datasets affecting peak estimation.
• Z-score mildly under-predicted meniscal volume; Nyúl was nearly unbiased. Z-score under-predicted volume by a mean difference of −103 mm³ (approximately 1,300 voxels), about 2% of total volume, with most of the error falling on meniscal boundaries. The authors highlight a qualitative example where a gap in the lateral posterior horn appears under min-max, robust min-max, and GMM normalisation but not with Nyúl.
• A qualitative failure mode is shown, not just aggregate metrics. In the case with the greatest Dice variability, several methods introduced a gap in the lateral posterior horn that was not present in the ground truth, and Nyúl did not.
• Histogram matching did not translate into segmentation gains. Although Nyúl aligned intensity distributions across datasets more closely than Z-score, this did not produce better segmentation, which the authors suggest may come at the cost of preserving biological variation.
• Domain shift from protocol and population differences is the dominant factor. Both datasets used DESS-type sequences but with different protocols (e.g., echo time, repetition time), and the populations differed (IWOAI 2019: patients at risk of or with early-stage osteoarthritis; SKM-TEA: patients with knee pathology). However, within the external dataset, Dice did not differ significantly between images with and without recorded pathology labels (Mann–Whitney U test, p = 0.67), so pathology presence alone does not explain the gap.

Methodology in Plain English

The researchers took an established 3D segmentation pipeline (nnU-Net, a framework that builds a 3D U-Net configuration automatically) and held everything constant except one variable: how image intensities are rescaled before the model sees them. They ran this pipeline seven times, once per normalisation method, using the same 5-fold cross-validation splits with a batch size of 2, patch size of 112 × 128 × 160, and 500 training epochs.

Training and internal testing used the IWOAI 2019 knee MRI dataset — 176 3D sagittal DESS images from 88 patients at two time points, sized 384 × 384 × 160 with 0.36 mm × 0.36 mm × 0.7 mm resolution, with train/validation/test folds of 120, 24, and 24. Images were cropped in-plane from 384 × 384 to 200 × 256, with the crop region chosen from the spatial distribution of ground-truth masks in the training data.

External testing used SKM-TEA — 3D qDESS knee scans from 155 patients acquired on two scanners at Stanford Healthcare, sized 512 × 512 × the number of slices (80 to 88 depending on knee size) at 0.31 mm × 0.31 mm × 0.8 mm. Because qDESS produces two echoes, these were combined with the root-sum-of-squares method, then cropped to 266 × 341 using the same width-to-height proportions as training.

Performance was measured with Dice Similarity Coefficient for overlap, CCC and MAE for volume agreement with manual annotations, and HD95 for boundary distance. Because Dice and HD95 vary far more between test images than between methods, the authors fitted separate linear mixed-effects models in R's lme4 package for each metric, with method as a fixed effect and random intercepts for fold and test image. Z-score was the reference category. A Type III ANOVA with Satterthwaite's method tested for an overall method effect; where significant, Tukey-adjusted pairwise comparisons followed, with α = 0.05 throughout. The five fold-trained models were also ensembled by averaging voxel-wise softmax probabilities for the box-plot visualisations.

Why This Matters

Impact on research. The paper reframes a common assumption in medical imaging preprocessing. Practitioners often treat normalisation as a lever for cross-site robustness, but this benchmark suggests that within musculoskeletal MRI the lever moves results by roughly 1% Dice while domain shift moves them by roughly 10%. It also provides a well-controlled protocol — identical folds, mixed-effects modelling, both volumetric and distance metrics — that other normalisation comparisons in other anatomies can reuse. Importantly, the authors position their methods as requiring no target-domain data and no retraining, in contrast to harmonisation approaches such as ComBat, adversarial domain adaptation, or style transfer, which typically need access to target data or task-specific adaptation.

Real-world applications:

• Multi-centre clinical deployment of segmentation models, where a model trained at one hospital is used on scans from another scanner or protocol without local annotation or fine-tuning.
• Osteoarthritis research and meniscus morphometry, where meniscal volume and boundary measurements feed into longitudinal studies of disease progression.
• Preprocessing pipeline design for musculoskeletal MRI, guiding teams that need to choose a normalisation step with limited evidence and want to know whether that choice is worth tuning.
• Benchmarking and reporting practice, giving model developers a template for quantifying whether a preprocessing choice matters relative to dataset-level effects rather than reporting it in isolation.

Industry relevance. Medical imaging software vendors and radiology AI companies face the practical problem that a cleared model must work on scanners and protocols it was not trained on. This paper quantifies how much of that problem is solvable with a cheap, retraining-free preprocessing change (little, in this setting) and how much requires other strategies. It also matters for regulatory and validation arguments: demonstrating robustness has to rest on external data, and the gap between internal (approximately 89% Dice) and external (approximately 79% Dice) results is a concrete illustration of why internal validation alone is insufficient.

Future Directions

• The inter-annotator variability question the paper itself flags as unresolved. Because meniscus segmentation is difficult, annotation differences between IWOAI 2019 (a single expert) and SKM-TEA (two researchers supervised by radiologists) could contribute to the external drop, but the authors state this is hard to assess and is a limitation of the study.
• What actually closes the ~10% domain-shift gap, given that normalisation accounts for only 0.5% of external variance. The paper frames simple normalisation as a practical baseline, leaving the comparative value of harmonisation and domain adaptation methods — which require target-domain access — as an open comparison.
• Building a knee-MRI equivalent of white stripe normalisation. The paper notes that no such knee-specific method exists, and that the GMM adaptation used here degraded under domain shift because intensity profiles differ between datasets. A knee-specific, anatomy-anchored standardisation method is an obvious next step.
• Disentangling which component of domain shift dominates. The paper isolates protocol differences, population differences, and pathology prevalence as candidates, ruling out pathology labels alone (p = 0.67), but its design cannot separate protocol effects from population effects any further. The authors cite related work on the same two datasets finding that sequence changes caused larger drops than scanner or centre effects.

Target Audience

Researchers and engineers working on medical image segmentation who deal with cross-site or cross-scanner deployment, particularly in musculoskeletal and orthopaedic imaging. It is also useful for radiomics and image-analysis methodologists interested in preprocessing effects, for clinical translation teams weighing how much effort to invest in normalisation versus other robustness strategies, and for anyone designing benchmarking studies who wants an example of mixed-effects modelling applied to fold-level model outputs. Readers looking for a new segmentation architecture or a new normalisation algorithm will not find one here; the contribution is the systematic comparison and the sizing of effects.

Authors’ abstract

Robust out-of-the-box performance is essential for the clinical deployment of deep learning models in medical imaging. An important but underexplored factor affecting model generalisability is intensity normalisation, particularly for magnetic resonance imaging (MRI), where image intensities vary across scanners and protocols. In this study, we systematically compared seven normalisation methods and their impact on the performance of a 3D U-Net model for meniscus segmentation from knee MRI. The methods included standard scaling approaches, histogram-based techniques, and a Gaussian Mixture Model (GMM)-based method. Models were trained on the IWOAI 2019 dataset and evaluated on both internal and external test sets (SKM-TEA) to assess generalisability. Performance was similar internally but differences were significant on external data, with Z-score, Nyúl histogram matching, and CLAHE showing greater robustness than other methods. However, these differences were small compared to the significant performance drop observed between datasets. Overall, while intensity normalisation had a measurable effect on model generalisability, its impact was limited relative to the effects of domain shift, highlighting the need for complementary strategies for robust deployment.