MieDB-100k: A Comprehensive Dataset for Medical Image Editing

AI summary

Overview

Research area: Computer vision / medical image analysis — specifically text-guided multimodal image editing and the datasets used to train and benchmark such models.

Technical level: Intermediate. Readers need some familiarity with diffusion-based image editing architectures (DiT backbones, LoRA finetuning, inpainting), segmentation metrics (DICE, PSNR, SSIM), and vision-language model evaluation, but the paper's central argument is data-centric and accessible without deep architectural knowledge.

Scope: The paper introduces and validates MieDB-100k, a 104,267-triplet text-guided medical image editing dataset spanning 10 imaging modalities and 63 editing targets, organized into Perception, Modification, and Transformation task categories.

What This Paper Is About

Medical image editing — instructing a model in natural language to, for example, remove a lesion or enhance a low-dose scan — is held back less by model architecture than by the near-total absence of suitable training and evaluation data. Existing medical editing datasets are narrow (often only chest X-rays and CTs), ignore medical image understanding in favor of pure generation, and either sacrifice clinical accuracy for cheap automated scaling or sacrifice scalability for manual collection. The authors build MieDB-100k to address all three gaps at once, and demonstrate that finetuning a general-purpose editing model on it produces state-of-the-art medical editing performance.

Key Contributions

1. The dataset itself. MieDB-100k contains 104,267 image-editing triplets, each of the form (input image I, textual prompt P, target image O), covering 63 distinct editing targets across 10 medical image modalities, with a documented, credible, and scalable construction pipeline.

2. Unifying medical image understanding and generation under a single "edit" paradigm. The authors are the first, by their account, to cast Perception tasks (localizing organs, lesions, or whole anatomical structures) as editing operations, and they show that joint training across Perception, Modification, and Transformation yields gains on specific tasks.

3. A three-perspective task taxonomy. Editing tasks are categorized as Perception (pixel-wise identification of prompted clinical targets), Modification (semantically altering medical content, e.g. lesion removal), and Transformation (holistic low-level operations such as restoration and enhancement), covering 11 distinct transformation targets under four low-level vision categories.

4. Benchmarking and ablation evidence. Nine existing models (six open-source, three closed-source) are evaluated on the benchmark, and an OmniGen2 baseline finetuned for 20,000 iterations (global batch size 64, learning rate 1e-4) is shown to surpass or match proprietary systems including Nano Banana Pro.

Main Findings

• Existing models fail at medical perception. On the Perception perspective, all tested models except the authors' finetuned version score at or below 0.467 DICE and at or below 0.221 Perception Accuracy (threshold τ = 0.8). Imagen4 and SDXL-turbo score 0.000 Perception Accuracy; OmniGen2 scores 0.248 DICE and 0.065 Perception Accuracy.

• The finetuned model dominates across all three perspectives. OmniGen2-MIE (7B) reaches 0.831 DICE, 0.737 Perception Accuracy, 28.1 background PSNR, 0.917 background SSIM, a 65.9 rubric score, 1.4 human preference rank, 22.2 PSNR, and 0.668 SSIM — best in every column.

• The strongest closed-source model shows the "right-for-the-wrong-reason" pattern. Nano Banana Pro achieves a competitive 63.4 rubric score and 2.0 preference rank in Modification, but only 0.426 DICE and 0.202 Perception Accuracy, indicating its edits are not backed by genuine medical localization knowledge.

• Nano Banana Pro is competitive on Transformation tasks. This is attributed either to similarity between tasks like denoising or artifact removal and general-purpose low-level vision, or to similar medical processing tasks having been in its training set; the authors still judge its absolute performance insufficient for clinical deployment.

• Joint training beats specialized training. In ablation, P-only training yields 0.833 DICE and 0.740 Perception Accuracy but only 37.8 rubric score; M-only training yields 57.5 rubric score but 0.001 DICE and 0.000 Perception Accuracy; T-only yields 23.9 PSNR and 0.696 SSIM but 15.0 rubric score. Training on the full MieDB-100k yields 0.831 DICE, 0.737 Accuracy, 65.9 rubric score, 22.2 PSNR, and 0.668 SSIM — matching or exceeding each specialist while improving Modification substantially.

• Generalization to unseen tasks. In an out-of-distribution test on bone metastasis addition and removal in CT scans — a target present in Perception data but strictly excluded from Modification training — OmniGen2-MIE substantially outperforms the untrained OmniGen2. Nano Banana Pro achieves the best OOD performance, marginally surpassing OmniGen2-MIE.

• Automated evaluation agrees with clinicians. Pairwise Spearman correlations among three evaluators are 0.963, 0.989, and 0.957 (all p < 1×10⁻⁵). Correlation between human preference ranking and the GPT-5.2 rubric score is ρ = −0.905 (p = 3×10⁻⁴).

• Data quality checks. Over 76% of reviewed Modification samples received unanimous 3-out-of-3 agreement among three board-certified medical experts. Of 6,000 randomly sampled training triplets, over 95% were rated high quality. Reconstruction of binary masks from colored overlays achieves average DICE of 0.999 for grayscale and 0.970 for RGB images.

Methodology in Plain English

The dataset is built differently for each of the three task families:

Perception is rule-based and scalable. Starting from existing segmentation datasets, the input image is the original scan, and the target image is the same scan with the ground-truth segmentation label overlaid in a randomly chosen color (red, green, or blue) at 0.6 alpha-blending transparency. The text prompt names the target and the color. Because it is a pure rule, it can be scaled by drawing on the large existing body of medical segmentation research. Manual filtering removes trivial, redundant, or mislabeled samples.

Modification is the hard case, because counterfactual pairs (the same patient with and without a lesion) cannot be captured in reality. Rather than distilling a general-purpose model — which hallucinates in medical settings — the authors train modality-specific "healthy tissue inpainting" experts via LoRA finetuning of FLUX.1-Fill-dev on non-pathological samples only, generating one expert each for CT, Ultrasound, MRI, Xray, Dermoscopy, and Endoscopy. They then take lesion-bearing images, mask the lesion with white pixels using its ground-truth segmentation, and have the matching expert synthesize a healthy replacement. Background restoration and edge blending correct any unintended FLUX modifications outside the mask. A rejection-sampling stage filters the results: for natural-image-like modalities (endoscopy, dermoscopy), Qwen3-VL-32B-Instruct judges a three-panel collage; for other modalities, separately trained nnUNet segmentation models discard any output where lesions remain detectable. Finally, triplets are formed by swapping input and output roles and varying prompts.

Transformation is assembled from public repositories as image pairs representing 11 transformation targets, with hand-designed prompts that map each low-level operation into the editing framework.

Post-processing. Prompts are rephrased by Qwen-Max for linguistic diversity. Training and test splits follow the source datasets strictly to preclude leakage. For training, images are resized into three resolution bins (128, 256, 512). The benchmark split consists of 3,397 representative, high-clinical-fidelity samples curated by three people with clinical backgrounds, kept at original image size.

Evaluation. Perception and Transformation use verifiable metrics (DICE, Perception Accuracy at τ = 0.8, PSNR, SSIM, with background-only variants for Perception). Modification uses a rubric-based VLM score from GPT-5.2 mapped to [0, 100], plus human preference ranking by three clinically trained evaluators.

Why This Matters

Impact on research. The paper reframes a data-scarcity problem as the core bottleneck for medical generative modeling, and provides both the artifact and a reproducible pipeline. Its central empirical claim — that training visual understanding alongside generation produces a synergistic gain — is a testable hypothesis the community can now probe at scale. The demonstration that a 7B finetuned open model beats proprietary systems on this benchmark also challenges the assumption that scale alone solves medical domain adaptation.

Real-world applications:

• Patient communication. Perception editing lets a patient point-and-ask "highlight the region you're describing," producing an annotated version of their own scan.
• Medical education. Students can prompt a model to localize or remove structures to build intuition about anatomy and pathology.
• Clinical triage efficiency. Automatically highlighted regions of interest could reduce screening time for senior clinicians.
• Low-level scan cleanup. The Transformation tasks cover restoration, enhancement, denoising, and artifact removal — pre-processing steps that feed downstream diagnostic pipelines.

Industry relevance. The dataset originates from DAMO Academy (Alibaba Group) with academic partners at Peking University, and its competitive results against Nano Banana Pro (Gemini-3-Pro-Image), GPT-Image-1, and Imagen4 make the case that domain-specific data, not proprietary scale, is the binding constraint. The inclusion of explicit human-inspection checkpoints and a VLM rubric that correlates strongly with clinician ranking (ρ = −0.905) gives regulated industries a template for auditable data curation and evaluation. The automatic filtering preservation rates reported per modality (86.1% CT, 76.1% Ultrasound, 61.6% MRI, 88.2% Xray, 64.9% Dermoscopy, 67.6% Endoscopy; the manual-inspection row is only partially present in the provided content) indicate the pipeline is not free, and MRI in particular demands substantial downstream culling.

Future Directions

• Broadening modality coverage. The authors explicitly list the inability to capture all medical imaging modalities as a limitation; the 10 covered modalities leave many clinical settings unaddressed.
• Rare disease representation. The scarcity of rare clinical cases is named as a limitation, raising the question of how to construct counterfactuals for pathologies with too few source images to train a reliable inpainting expert.
• Extending beyond editing. The current scope is editing tasks alone; incorporating classification, report generation, or diagnostic reasoning under the same unified paradigm is an open direction.
• Closing the gap to scaled proprietary models. Because Nano Banana Pro marginally outperformed OmniGen2-MIE on the OOD bone metastasis task, the authors conclude that further scaling of medical editing data — not just better curation — is needed to match proprietary models.

Target Audience

This paper is most valuable to researchers and engineers building or evaluating multimodal generative models for clinical use, particularly those working on domain adaptation, medical image synthesis, or dataset construction. It also serves medical AI regulators and quality teams interested in a concrete template for human-in-the-loop data validation, and benchmarking researchers who need a diverse, multi-modality test bed spanning both understanding and generation. Clinicians with an interest in AI tooling will find the Perception task framing directly relevant to their workflows.

Authors’ abstract

The scarcity of high-quality data remains a primary bottleneck in adapting multimodal generative models for medical image editing. Existing medical image editing datasets often suffer from limited diversity, neglect of medical image understanding and inability to balance quality with scalability. To address these gaps, we propose MieDB-100k, a large-scale, high-quality and diverse dataset for text-guided medical image editing. It categorizes editing tasks into perspectives of Perception, Modification and Transformation, considering both understanding and generation abilities. We construct MieDB-100k via a data curation pipeline leveraging both modality-specific expert models and rule-based data synthetic methods, followed by rigorous manual inspection to ensure clinical fidelity. Extensive experiments demonstrate that model trained with MieDB-100k consistently outperform both open-source and proprietary models while exhibiting strong generalization ability. We anticipate that this dataset will serve as a cornerstone for future advancements in specialized medical image editing.