Deep infant brain segmentation from multi-contrast MRI

AI summary

Overview

Research area: Machine learning for medical image analysis — specifically deep-learning brain MRI segmentation in infants and young children (pediatric neuroimaging).

Technical level: Intermediate (readers should be comfortable with convolutional neural networks, U-Nets, and basic MRI contrast concepts).

Scope: The paper presents BabySeg, a single deep-learning framework that segments infant and pediatric brain MRI across a wide age range and a flexible number of input scans — including image types never seen during training — combining a group-convolutional U-Net with a domain-randomization training data engine.

What This Paper Is About

Automatically outlining brain structures on MRI is hard in infants and young children: scans are difficult to acquire, often contain a lot of non-head anatomy from the torso and limbs, contain motion artifacts, and — because of progressive myelination — the image contrast itself flips over the first years of life, passing through a low-contrast "isointense phase." Existing tools tend to be narrowly scoped to one modality, one age window, or one image type. The goal of this work is a single model that handles diverse MRI protocols, varying numbers of input scans, and image types not available during training.

Key Contributions

1. A domain-randomization training data engine that mixes synthetic and real images, enabling generalization to unseen image types while still learning precise signals from real scans.
2. Demonstrated state-of-the-art accuracy from extremely preterm birth across the first five postnatal years using one model, verified on cohorts spanning that range.
3. A group convolution mechanism that lets the network accept a flexible number of input scans of arbitrary type and size, by separately convolving each input and also convolving the mean of the group to allow cross-input interaction.
4. Large runtime savings: 11- to 24-fold speed-ups over baselines for T1w–T2w segmentation on limited (single-threaded CPU) hardware.

Main Findings

• Broad accuracy across cohorts: BabySeg generally ranks among the two best-performing methods across test sets and input configurations, and avoids low-accuracy cases with mean Dice scores substantially less than 70 points, which each baseline produces in several test settings.
• Best on in-distribution data, still strong out-of-distribution: Dice scores are highest on the held-out in-distribution dHCP test set, but performance remains high out of distribution, such as for Infant FreeSurfer subjects whose scan age exceeds the oldest training subjects by up to 4 years.
• Handles extreme ages: Qualitatively, BabySeg generalizes to a severely preterm neonate scanned at 26 weeks postmenstrual age, where baselines substantially over- or undersegment, and accurately segments the oldest IFS subject scanned at 4.7 years. BIBSNet undersegments that subject, likely because brain size exceeds the field of view of its internal registration template.
• Additional input modality helps slightly: Supplying both T1w and T2w inputs slightly improves median Dice scores across dHCP and M-CRIB — except for BIBSNet, which struggles with anterior and posterior commissure alignment across contrasts, a step users cannot disable.
• Runtime: On a single computational thread, Infant-SynthSeg averages 56 minutes per T1w–T2w pair (most spent on postprocessing), and BIBSNet needs roughly half that but still over 10 minutes even for a single modality. BabySeg runs in about 2 minutes, a greater than 11-fold speed-up over BIBSNet and 24-fold over Infant-SynthSeg. On GPU, BabySeg takes 2 seconds per image with 20 seconds of one-time setup, equivalent to 1,800 segmentations per hour.
• Stated limitations: BabySeg does not delineate the vermis, ventricular subtypes, or cerebrospinal fluid; it assumes input images are already registered to one another; and processing all inputs in parallel limits segmentation of subjects with many modalities to systems with sufficient memory.

Methodology in Plain English

BabySeg is built from two parts.

The network. Instead of stacking multiple scans as channels — which fixes how many inputs a model can take — the authors use a "group convolution." Each input scan is convolved separately, and the average of all the inputs is convolved as well; the two results are combined (with a 0.5 weight on each, plus a bias) so information can flow between scans. This means the layer works for any number of inputs. The overall architecture is a U-Net with 6 downsampling levels, two 3×3×3 convolutions per level with ELU activations and max pooling, and a mirrored upsampling arm with skip connections. All convolutions use 64 filters, except the first level, which derives a single feature, and the final convolution, which outputs K = 22 softmax-activated maps. Features are fused across inputs by averaging the nonlinearly activated outputs of the second-last layer.

The training data engine. At each step, the engine samples a label map and its associated registered images from one imaging session, then: (1) randomly translates, rotates, scales, shears, and nonlinearly warps the anatomy, while filling the label-map background with 0 to 3 synthetic geometric "blobs" derived from thresholded gradient noise; (2) synthesizes 1 to 4 images with artificial contrasts — some are real scans passed through a randomized smooth intensity lookup table, others are generated directly from the label map by assigning random intensities to each anatomical label; and (3) applies a randomized series of corruptions including bias-field modulation, partial voluming, Gaussian noise, gamma transformation, resolution reduction, constant-intensity slice replacement, cropping, and simulated skull-stripping with over- and undersegmentation. All randomization ranges are listed in Table I.

Training resamples everything to 192×192×192 isotropic 0.7-mm voxels in left-inferior-anterior orientation, clips intensities outside the 1–99 percentile, and rescales to [0,1]. The loss is a Dice-based overlap criterion. Optimization uses Adam with batch size 1, an initial learning rate of 10⁻⁴ reduced to 10⁻⁵, for a total of at least 1 million steps, on an Nvidia A100 GPU with 40 GB memory.

Evaluation. Four datasets are used: BOBs (71 subjects, T1w and T2w, aged 1–9 months), M-CRIB (10 subjects, 40–43 weeks postmenstrual age), dHCP release 4 (782 term and preterm newborns, 709 T1w and 885 T2w images, 27–45 weeks postmenstrual age), and IFS (15 subjects, 0–5 years, T1w). Four BOBs and four dHCP subjects are held for validation and 20 dHCP subjects for testing, spread evenly across the postmenstrual age range; M-CRIB and IFS are used exclusively for out-of-distribution testing. Label maps are remapped to the largest possible set of 21 overlapping FreeSurfer structures, plus background, giving 22 output labels. Baselines are Infant-SynthSeg and BIBSNet, giving six baseline setups across input contrasts.

Why This Matters

Impact on research. Pediatric neuroimaging has fragmented into many narrow tools, each tied to a specific modality or age window. BabySeg argues that a single model trained with aggressive randomized synthesis can cover extremely preterm neonates through five years of age, which simplifies comparison across studies and reduces the need to select and maintain separate pipelines.

Real-world applications:

• Automated morphometry for characterizing individual brain growth and maturation in infants and young children.
• Studying developmental disorders by enabling consistent structural measurements across cohorts of differing ages and scan protocols.
• Population-level analyses that require processing thousands of scans where runtime is a practical bottleneck.
• Downstream analyses such as fitting cortical surface meshes, which depend on accurate initial segmentation.
• Supporting dependent analyses for scans that are messy in the ways clinical scans typically are — motion artifacts, incomplete protocol coverage, and non-head anatomy in the field of view.

Industry relevance. The runtime results matter for deployment: the paper contrasts roughly 2 minutes for BabySeg against 56 minutes for Infant-SynthSeg on a single CPU thread, and notes a GPU throughput equivalent to 1,800 segmentations per hour. The ability to accept whatever scans are available, rather than requiring a fixed modality set, fits real clinical workflows where a session may be cut short by patient motion and the planned sequences are never fully collected.

Future Directions

• Extending the label set: adding the vermis, ventricular subtypes, and cerebrospinal fluid, which BabySeg currently does not delineate.
• Removing the registration assumption: the current framework assumes prior registration of input images; the authors suggest simulating alignment errors within the data engine or predicting a separate label map per input.
• Scaling to many modalities: the present implementation processes all inputs in parallel, limiting subjects with many modalities to systems with sufficient memory; a more memory-efficient scheme could relax this.
• Wider age and protocol coverage: the authors note that spatial augmentation already allows inference with a field of view larger than training, and the out-of-distribution IFS results suggest the approach could be pushed further beyond the trained age range.

Target Audience

Machine learning researchers working on medical image segmentation and domain randomization; neuroimaging scientists and developers building pediatric processing pipelines; and clinical researchers who need reliable, fast brain segmentation for infant and early-childhood MRI cohorts. Those interested in flexible-input architectures, synthetic-data training strategies, or practical runtime comparisons between segmentation tools will find the contributions directly relevant.

Authors’ abstract

Segmentation of magnetic resonance images (MRI) facilitates analysis of human brain development by delineating anatomical structures. However, in infants and young children, accurate segmentation is challenging due to development and imaging constraints. Pediatric brain MRI is notoriously difficult to acquire, with inconsistent availability of imaging modalities, substantial non-head anatomy in the field of view, and frequent motion artifacts. This has led to specialized segmentation models that are often limited to specific image types or narrow age groups, or that are fragile for more variable images such as those acquired clinically. We address this method fragmentation with BabySeg, a deep learning brain segmentation framework for infants and young children that supports diverse MRI protocols, including repeat scans and image types unavailable during training. Our approach builds on recent domain randomization techniques, which synthesize training images far beyond realistic bounds to promote dataset shift invariance. We also describe a mechanism that enables models to flexibly pool and interact features from any number of input scans. We demonstrate state-of-the-art performance that matches or exceeds the accuracy of several existing methods for various age cohorts and input configurations using a single model, in a fraction of the runtime required by many existing tools.