Rethinking Bias in Generative Data Augmentation for Medical AI: a Frequency Recalibration Method

AI summary

Overview

Research area: Medical AI, generative data augmentation (GDA), and frequency-domain image analysis. The paper sits at the intersection of synthetic medical image generation (GANs and diffusion models) and downstream medical image classification.

Technical level: Intermediate. The core intuition is accessible, but the method relies on Fourier spectra, Gaussian modeling of frequency bands, and a transformer-based denoising auto-encoder.

Scope: One sentence: the paper argues that the unreliability of AI-synthesized medical images stems largely from a high-frequency mismatch with real images, and proposes a two-step post-processing method, Frequency Recalibration (FreRec), that closes that gap and improves downstream classification across brain MRI, chest X-ray, and fundus image tasks.

What This Paper Is About

Medical AI needs large datasets but suffers from data scarcity, so researchers synthesize images with generative models to augment training sets (GDA). The problem is that these synthetic samples do not always help and can even hurt downstream classifiers — a risk the authors say is underestimated in the medical domain. The paper traces this unreliability to a frequency gap between real and AI-generated images and proposes FreRec to reduce it.

Key Contributions

1. Diagnosis of the bias source. The authors identify frequency misalignment between real and synthesized medical images as one of the key factors behind unreliable GDA, showing empirically that plain GDA sometimes degrades classifiers rather than improving them.
2. The FreRec method. A two-stage, coarse-to-fine calibration: (1) Statistical High-frequency Replacement (SHR), which replaces the high-frequency components of a synthetic image with statistically sampled counterparts from similar real images; and (2) Reconstructive High-frequency Mapping (RHM), which restores image quality and high-frequency detail by projecting synthetic images onto a latent natural frequency manifold learned only from real images.
3. A model-agnostic, plug-and-play design. Unlike prior work that requires retraining the generative model with frequency-domain regularization, FreRec is a standalone post-processing step compatible with any generative model (GANs and diffusion models) and requires no access to the generator.
4. Broad empirical validation. Experiments across three medical imaging modalities and three classifier backbones — plus an extension to CIFAR-10 (Automobile, Ship, Truck) — with frequency-distribution and T-SNE feature-space visualizations before and after recalibration.

Main Findings

• Plain GDA is unstable in medical imaging. GDA decreases accuracy and F1 scores of cardiomegaly-ResNet50 and of all brain tumor classifiers compared to raw training. For example, on brain tumor MRI with ResNet50, GDA yields AUC 0.783 / Acc 0.723 / F1 0.723 versus RAW at 0.793 / 0.783 / 0.772.
• FreRec consistently recovers and improves performance. GDA+FreRec improved over raw classifiers in all tests and metrics. Selected results: cardiomegaly with DenseNet AUC 0.899 / Acc 0.848 / F1 0.834 (vs RAW 0.842 / 0.803 / 0.787); cardiomegaly with ResNet50 0.888 / 0.832 / 0.834 (vs RAW 0.834 / 0.792 / 0.788); brain tumor with ResNet50 0.843 / 0.843 / 0.783 (vs RAW 0.793 / 0.783 / 0.772); DR with ResNet50 0.878 / 0.823 / 0.813 (vs RAW 0.843 / 0.792 / 0.783).
• One exception to the win. GDA+FreRec outperformed the baselines in all tasks except against DoGE on DR classification (DoGE: DenseNet 0.880 AUC / 0.841 Acc / 0.816 F1; ResNet50 0.871 / 0.825 / 0.819; ViT 0.862 / 0.811 / 0.810). The authors attribute this to DoGE relying on a fixed, known synthetic image source, whereas FreRec is trained only on real images.
• Frequency distributions align after recalibration. Before FreRec, substantial frequency discrepancies existed in all datasets despite three different modalities and three different generators; after recalibration the gaps were reduced in all datasets. Alignment was more thorough for Brain Tumor MRI and Cardiomegaly X-ray than for DR fundus images — the authors suggest this is because DR fundus images are color photographs with richer pixel information, which is harder for the auto-encoder to learn.
• Feature-space bias is largely removed. T-SNE visualizations show synthetic and real features clearly separated before recalibration. After recalibration, features completely overlap for Brain Tumor MRI and Cardiomegaly X-ray; for DR the shift is incomplete, again attributed to richer color pixel information and other contextual biases.
• Both steps are necessary. In the ablation, SHR alone does not improve classification despite roughly aligning frequencies, because it significantly distorts image quality (Card: AUC 0.81 / Acc 0.79, PSNR 25.10, SSIM 0.76). Adding RHM is essential: FreRec reaches Card AUC 0.89 / Acc 0.84, PSNR 35.62, SSIM 0.95; DR 0.87 / 0.82, PSNR 34.29, SSIM 0.95; BT 0.83 / 0.81, PSNR 41.15, SSIM 0.98.
• The FESA module trades image quality for classification gains. RHM without FESA produces the best reconstruction quality (Card PSNR 36.44, SSIM 0.98; DR 36.33 / 0.97; BT 41.23 / 0.99) due to its 1:1 reconstruction, but the full FreRec with FESA achieves higher classification and is only marginally behind in image quality.
• Low inference overhead. Per-image inference time on the stated NVIDIA GTX 4090 server: 16.17 ms for chest X-ray, 15.04 ms for fundus images, 17.63 ms for brain MRI — faster than DoGE (33.21 ms, 36.14 ms, 33.78 ms respectively) on the same three tasks.
• The Gaussian assumption holds. Sampling 1,000 images from each dataset, the spectral amplitude histograms at bands k = 60, 80, and 100 approximated a Gaussian distribution with an average skewness of −0.835.
• Non-medical generalization. On CIFAR-10 (StyleGAN2 pretrained by StudioGAN; Automobile, Ship, Truck; 5,000 real and 5,000 synthetic training images per class, 1,000 real test images per class), GDA was more stable than in the medical setting, but FreRec still improved accuracy from about 0.945 to 0.980 (ResNet18 0.973, ResNet50 0.978, DenseNet 0.981).

Methodology in Plain English

The starting observation is that real and AI-generated images differ in their higher-frequency content — the fine-grained texture and edge information — and that medical images such as MRIs and X-rays are especially sensitive to those frequencies because diagnosis relies on subtle pathological detail.

FreRec fixes this in two steps.

Step one — Statistical High-frequency Replacement (SHR). Each synthesized image is converted into the frequency domain and split into a low-frequency part and a high-frequency part using a binary mask with a fixed ratio r. To avoid a random one-to-one swap, the method retrieves the top-K real images that are most similar to the synthetic image by SSIM score, models their high-frequency statistics as a Gaussian distribution, samples a new mean and standard deviation from it, and re-normalizes the synthetic image's high-frequency component to those statistics. Combining this back with the original low-frequency component and inverting the transform produces a roughly calibrated but visually degraded image — a "lossy" sample.

Step two — Reconstructive High-frequency Mapping (RHM). The degraded sample is passed through a denoising auto-encoder trained exclusively on real images. The key design choice is that the auto-encoder's real-image inputs are perturbed with the same SHR procedure, so real and synthetic images start from the same space and follow the same reconstruction path. Because the mapping is unidirectional — synthetic images are projected onto a latent natural frequency manifold learned from real data — the reconstruction does not simply revert to the original misaligned synthetic image.

The auto-encoder architecture. The network is based on the Restoration Transformer (Restormer), with its transformer blocks replaced by Frequency-enhanced Transformer blocks (FET-blocks). The architecture uses 2, 4, 6, and 8 FET-blocks in levels 1 through 4, plus two more in a refinement stage. Each block combines a global spatial self-attention branch operating on RGB features with a local frequency self-attention branch that runs FFT on the features, partitions the amplitude spectrum into non-overlapping circular rings of width d along the radius, groups features in the same ring into one channel, applies convolutions and a sigmoid-based local attention map, then inverse-FFTs and fuses with the RGB branch. Training uses a joint loss combining pixel-level similarity and spectral similarity.

Deployment recommendation. Because the trained auto-encoder is imperfect, small frequency differences can remain. The authors therefore recommend applying FreRec as a unified pre-processing module in both the training and inference phases of the downstream classifier, so that all real and synthetic samples share a common frequency distribution.

Evaluation setup. Three medical tasks: brain tumor detection from brain MRIs (Kaggle-Brain tumor dataset, FastGAN pretrained), cardiomegaly diagnosis from chest X-rays (MIMIC-CXR, StyleGAN3 trained from scratch), and diabetic retinopathy classification from fundus photography (Kaggle-DR, VC-Diffusion pretrained). Classifiers: ResNet50, DenseNet, and ViT-B-16, each evaluated five times with averaged Acc, F1, and AUC. Comparisons: RAW (no augmentation), GDA without alignment, GDA+FreRec, AutoAug, Mix-up, AFA (Fourier-basis Augmentation), and DoGE. The SHR mask ratio r was set to 0.5 and the sample number k to 200, chosen as a trade-off between alignment effect, image quality, and running time.

Why This Matters

The paper challenges a common assumption in medical AI: that adding synthetic images is automatically beneficial. It shows the opposite can happen, and that a cheap, model-agnostic post-processing fix can turn synthetic data into a reliable resource — without retraining expensive generative models or needing access to their internals. This is a practical result for anyone building medical imaging pipelines under data scarcity.

Real-world applications:

• Brain tumor detection from MRI, where GDA alone degraded every classifier tested and FreRec restored and exceeded raw-training performance.
• Cardiomegaly screening from chest X-rays, where a from-scratch StyleGAN3 pipeline benefited substantially from recalibration.
• Diabetic retinopathy screening from fundus photographs, where gains were real but smaller, indicating modality-specific limits.
• General medical or scientific data augmentation pipelines, since FreRec is a plug-and-play module that can sit in front of any downstream classifier and any generative model (the paper also reports benefits on CIFAR-10).

Industry relevance: Hospitals and medical AI vendors that want to expand scarce training data can adopt a lightweight post-processing step with per-image inference times of roughly 15–

Authors’ abstract

Developing Medical AI relies on large datasets and easily suffers from data scarcity. Generative data augmentation (GDA) using AI generative models offers a solution to synthesize realistic medical images. However, the bias in GDA is often underestimated in medical domains, with concerns about the risk of introducing detrimental features generated by AI and harming downstream tasks. This paper identifies the frequency misalignment between real and synthesized images as one of the key factors underlying unreliable GDA and proposes the Frequency Recalibration (FreRec) method to reduce the frequency distributional discrepancy and thus improve GDA. FreRec involves (1) Statistical High-frequency Replacement (SHR) to roughly align high-frequency components and (2) Reconstructive High-frequency Mapping (RHM) to enhance image quality and reconstruct high-frequency details. Extensive experiments were conducted in various medical datasets, including brain MRIs, chest X-rays, and fundus images. The results show that FreRec significantly improves downstream medical image classification performance compared to uncalibrated AI-synthesized samples. FreRec is a standalone post-processing step that is compatible with any generative model and can integrate seamlessly with common medical GDA pipelines.