Lesion-Aware Post-Training of Latent Diffusion Models for Synthesizing Diffusion MRI from CT Perfusion

AI summary

Overview

Research area: Medical image-to-image translation using generative models — specifically, latent diffusion models (LDMs) applied to synthesizing diffusion MRI from CT perfusion in acute ischemic stroke.

Technical level: Intermediate. Readers should be comfortable with diffusion models, latent spaces, and standard medical image evaluation metrics (MAE, PSNR, MS-SSIM, FID), though the paper explains its novel components in enough detail to follow.

Scope: The paper proposes a post-training framework that adds lesion-aware image-space objectives to a pre-trained latent diffusion model, and evaluates it on DWI and ADC synthesis from CT perfusion scans from 817 patients.

What This Paper Is About

Latent diffusion models generate images efficiently by working in a compressed latent space, but this compression can lose fine pixel-level detail — a serious problem when the structure of interest, such as a stroke lesion, occupies only a small fraction of the image. The authors address this by post-training an already-trained latent diffusion model with two extra objectives computed in full image space: one for overall image fidelity and one that focuses only on ischemic lesion regions. The goal is CT-to-MRI translation for acute ischemic stroke, where accessible CT perfusion scans are used to synthesize diffusion MRI (DWI and ADC), the gold standard for stroke diagnosis.

Key Contributions

1. A post-training framework for LDMs with medical image-space objectives. Instead of freezing the decoder, the method adds image-space losses on top of the standard latent diffusion objective, using a one-step estimate of the noise-free latent to keep training efficient.

2. A lesion-aware loss. A binary lesion mask is applied to the image-space reconstruction error, so the model receives concentrated gradient signal on ischemic lesion voxels rather than having that signal diluted by the surrounding parenchyma and background.

3. State-of-the-art results on brain CT-to-MRI translation. Using a paired dataset of 817 acute ischemic stroke patients, the post-trained model (cLDM-PT) achieves the lowest MAE, lowest lesion MAE, highest MS-SSIM, and lowest FID among the compared models for both DWI and ADC synthesis.

4. Demonstration that the framework generalizes to other latent diffusion models. Applying the same post-training procedure to conditional BBDM, which learns a stochastic Brownian bridge in latent space, improved all reported metrics.

Main Findings

• Best overall quantitative results: cLDM-PT reached a DWI MAE of 0.072, lesion MAE of 0.199, PSNR of 27.78, MS-SSIM of 0.867, and FID of 29.95. For ADC it reached MAE 0.052, lesion MAE 0.105, PSNR 31.49, MS-SSIM 0.876, and FID 61.91 — the lowest MAE, lowest lesion MAE, highest MS-SSIM, and lowest FID in both columns.

• Improvement over the pre-post-training baseline: The base conditional LDM (cLDM) scored DWI MAE 0.073, lesion MAE 0.222, PSNR 27.94, MS-SSIM 0.855, FID 30.82, and ADC MAE 0.072, lesion MAE 0.125, PSNR 27.08, MS-SSIM 0.866, FID 66.71. The paper reports an overall 14.5% reduction in image MAE and 12.4% reduction in lesion MAE after post-training.

• Lesion delineation is the distinctive gain: Lesion MAE dropped to 0.199 for DWI and 0.105 for ADC, which the authors highlight as evidence of more precise lesion delineation. Qualitative figures show cLDM-PT depicting restricted-diffusion lesions that other models miss, along with better grey-white matter differentiation.

• GAN and diffusion baselines fell short: CycleGAN (DWI MAE 0.143, lesion MAE 0.235, FID 40.62), Pix2Pix (DWI MAE 0.083, FID 46.90), PairedCycleGAN (DWI MAE 0.123, FID 47.96), BBDM (DWI MAE 0.104, FID 31.84), and cBBDM (DWI MAE 0.093, FID 32.32) all produced less accurate or less perceptually convincing results. The authors note that diffusion-family models generate more realistic images than GANs but still struggle with lesion delineation on their own.

• Small loss weights work best: In the weight sweep (Table 2), increasing the image loss weight improved accuracy and structural consistency but with diminishing returns. Increasing the lesion loss weight sharply reduced lesion MAE until it plateaued, and further increases distorted the images overall. The selected best configuration was λ_image = 0.01 and λ_lesion = 0.02.

• Transferability to a second latent diffusion model: Post-training cBBDM improved it from DWI MAE 0.093, lesion MAE 0.226, PSNR 25.21, MS-SSIM 0.800, FID 32.32 to DWI MAE 0.078, lesion MAE 0.221, PSNR 27.40, MS-SSIM 0.857, FID 31.55, and from ADC MAE 0.098, lesion MAE 0.139, PSNR 23.63, MS-SSIM 0.777, FID 66.98 to ADC MAE 0.074, lesion MAE 0.120, PSNR 26.62, MS-SSIM 0.866, FID 66.67.

• Dataset characteristics: Slices containing lesions were a minority of the data — 10.9% (1542/14083) in training, 11.3% (220/1948) in validation, and 11.0% (441/4015) in test — with mean (±SD) lesion volumes of 15.17±41.17 ml, 12.46±32.94 ml, and 14.66±36.24 ml respectively, underscoring why a lesion-specific objective matters.

Methodology in Plain English

The starting point is a standard conditional latent diffusion model built on the VQGAN framework: an encoder compresses each MRI slice into a smaller latent representation, a time-conditional UNet learns to denoise in that latent space, and a decoder reconstructs images. The CT perfusion input is spatially rescaled and concatenated channel-wise as the conditioning signal. This base model is trained first with the conventional latent diffusion loss.

The new step is post-training. During training, instead of running the full iterative denoising chain, the model takes a noisy latent, produces a one-step estimate of the clean latent, and decodes it back to image space. Two additional losses are then computed there: a pixel-wise reconstruction loss over the whole image, and a masked reconstruction loss that only counts voxels inside the ischemic lesion mask. The total objective is the original latent loss plus these two, each scaled by a hyper-parameter. Because the lesion loss is restricted to the lesion mask, the gradient that tells the model "make the lesion correct" is not drowned out by the much larger number of normal brain and background voxels.

Training used the 817-patient SNUH dataset split into 571 training patients (14083 slices), 81 validation patients (1948 slices), and 165 test patients (4015 slices). MRI was acquired at 3.0T with in-plane voxel sizes of 0.9375×0.9375 mm to 1×1 mm and 4–5 mm slice thickness, resampled to 1×1×5 mm³. ADC maps came from DWI (b=1000) and were used by medical experts to create ground-truth lesion masks. CTP came from an Aquilion 64 CT scanner (TOSHIBA) at 0.47×0.47×1 mm³ across 15 time points, registered into DWI space with ANTs. The model used AdamW with a base learning rate of 2×10⁻⁶, T=1000 diffusion steps, and 200 DDIM sampling steps at inference, trained on an NVIDIA A6000 GPU with batch size 48.

Why This Matters

This work targets a real clinical bottleneck. CT is fast, cheap, and widely available, while diffusion MRI is far better at detecting acute ischemic stroke but limited by cost, access, scan duration, and patient contraindications. If CT perfusion can be translated into reliable synthetic DWI and ADC — including accurate lesion depiction — it could extend stroke diagnostic capability to settings that lack timely MRI access. The paper's broader point is that efficient latent-space generative models systematically under-serve small but clinically decisive structures, and that targeted post-training can correct this without retraining from scratch.

Research impact: The framework is presented as a general post-training recipe that can be applied to pre-trained LDMs, and the cBBDM experiment supports that claim. It connects to recent work on image-space post-training for diffusion models but extends it with a task-specific, region-masked objective.

Real-world applications:

• Emergency stroke triage, where early and accurate diagnosis determines treatment selection.
• Hospitals with CT but limited or overloaded MRI capacity.
• Reducing patient burden for those who cannot tolerate long MRI scans or have contraindications.
• Extending to other medical image translation tasks where the structure of interest is small, which the authors explicitly name as a broader opportunity.

Industry relevance: The code is publicly released at https://github.com/snuh-rad-aicon/Diffusion-LAPT, and the method is designed as a lightweight add-on to existing trained models rather than a new architecture, which lowers the barrier to adoption by medical imaging and clinical AI vendors. The work was supported by the National Research Foundation of Korea, the SNU-SNUH Physician Scientist Training Program, the SNUH Research Fund, and the Korea Health Industry Development Institute.

Future Directions

• Validation beyond a single institution and scanner. All data came from Seoul National University Hospital with specific scanners and acquisition protocols; external multi-site validation is a natural next step.
• Testing the framework on other modalities and lesion types. The authors state the approach has potential for broader medical image translation tasks, but only CT-to-MRI for ischemic stroke is tested here.
• Understanding the limits of the lesion loss weighting. The weight sweep shows that large lesion weights distort the whole image; how to select these weights reliably for new tasks without an extensive sweep is left open.
• Clinical evaluation of diagnostic value. The study reports reconstruction metrics and expert-facing visualizations, but whether synthetic DWI/ADC actually improves downstream clinical decision-making is not established in the reported content.

Target Audience

Medical image analysis researchers working on image-to-image translation and generative models; clinical AI developers interested in deploying low-cost imaging surrogates; radiologists and stroke clinicians curious about what synthetic diffusion MRI can and cannot yet deliver; and machine learning researchers studying how latent compression affects the fidelity of small, diagnostically critical structures.

Authors’ abstract

Image-to-Image translation models can help mitigate various challenges inherent to medical image acquisition. Latent diffusion models (LDMs) leverage efficient learning in compressed latent space and constitute the core of state-of-the-art generative image models. However, this efficiency comes with a trade-off, potentially compromising crucial pixel-level detail essential for high-fidelity medical images. This limitation becomes particularly critical when generating clinically significant structures, such as lesions, which often occupy only a small portion of the image. Failure to accurately reconstruct these regions can severely impact diagnostic reliability and clinical decision-making. To overcome this limitation, we propose a novel post-training framework for LDMs in medical image-to-image translation by incorporating lesion-aware medical pixel space objectives. This approach is essential, as it not only enhances overall image quality but also improves the precision of lesion delineation. We evaluate our framework on brain CT-to-MRI translation in acute ischemic stroke patients, where early and accurate diagnosis is critical for optimal treatment selection and improved patient outcomes. While diffusion MRI is the gold standard for stroke diagnosis, its clinical utility is often constrained by high costs and low accessibility. Using a dataset of 817 patients, we demonstrate that our framework improves overall image quality and enhances lesion delineation when synthesizing DWI and ADC images from CT perfusion scans, outperforming existing image-to-image translation models. Furthermore, our post-training strategy is easily adaptable to pre-trained LDMs and exhibits substantial potential for broader applications across diverse medical image translation tasks.