MTEdit: Packed Trajectories Beat Rollouts for Multi-Turn Image Editing

AI summary

Overview

Research area: Computer Vision — instruction-based image editing, with a focus on multi-turn (sequential) image editing, training-data construction, and benchmark design.

Technical level: Advanced. The paper assumes familiarity with diffusion/flow-based image editing models, vision-language models (VLMs), attention masking, and positional encoding schemes such as RoPE.

Scope: This paper introduces MTEdit, a 500K-trajectory multi-turn image editing dataset built with a "packed trajectory canvas" generation pipeline, plus MTEdit-Bench, a benchmark for step-wise and context-aware multi-turn editing, and Qwen-Image-MTEdit, a fine-tuned model trained on the dataset.

What This Paper Is About

Multi-turn image editing lets a user modify an image through a sequence of instructions, which matches how people actually iterate on creative work. Building training data for this setting is hard: the usual approach is to feed each edit's output into the next edit ("sequential rollout"), which lets errors and visual artifacts pile up, so image quality degrades as the chain gets longer. The paper's goal is to construct high-quality, long multi-turn editing data that avoids this drift, to build a benchmark that actually measures drift and history use, and to show the data improves a model when used for fine-tuning.

Key Contributions

1. MTEdit dataset: 500K multi-turn editing trajectories and 2.69M edit pairs, with 4 to 8 turns per trajectory and an average of 5.38 turns, covering 23 editing categories organized under 5 coarse groups (object structure, object appearance, scene composition, local detail, atmospheric edits). It is built with a packed trajectory canvas (PTC) generation pipeline rather than sequential rollout, which the authors show reduces visual-quality drift across turns.

2. Coverage of both editing paradigms: The dataset supports step-wise editing (56% of data) and context-aware editing (44%), the latter split into textual-reference (25%), short-instruction (12%), and persistent-constraint (7%) subsets. The authors state this is broader scenario coverage than compared datasets such as MagicBrush, SEED-Data-Edit, ImgEdit, and Pico-Banana-400K.

3. MTEdit-Bench: A benchmark with a step-wise sub-bench of 100 eight-turn samples and three context-aware sub-benches of 50 four-turn samples each, scoring instruction following (IF), content consistency (CC), a Drift Robustness Score (DRS), and context-aware variants IF* and CC*.

4. Qwen-Image-MTEdit: A model obtained by fine-tuning Qwen-Image-Edit-Causal on MTEdit, which the authors report as the best-performing open-source model among those evaluated on MTEdit-Bench, and which improves over its base model on IF, CC, and DRS across all eight turns.

Main Findings

• Dataset scale exceeds prior multi-turn datasets. MTEdit has 500K trajectories and 2.69M edit pairs with maximum 8 turns and average 5.38 turns, versus Pico-Banana-400K (77K trajectories, 306K edits, max 6 turns, average 3.97), ImgEdit (115K, 314K, max 3, average 2.73), SEED-Data-Edit (21K, 81K, max 6, average 3.82), and MagicBrush (3K, 8K, max 3, average 2.58).

• PTC generation beats sequential rollout on data quality. Across 5,000 matched eight-turn cases, the PTC pipeline's DRS stays between 4.27 and 4.30 over all eight turns, while rollout pipelines drop to 3.51 (Seedream 5.0 Lite), 3.83 (GPT Image 2), and 3.70 (Nano Banana 2) at Turn 8.

• Controlled ablation confirms the training benefit. With 10,000 eight-turn step-wise trajectories from identical source images and per-turn prompts generated by Nano Banana 2, and 5,000 fine-tuning steps on the same Qwen-Image-Edit-Causal base, PTC-trained data yields higher IF from Turn 2 onward, higher CC at every turn, and higher DRS from Turn 3 onward. At Turn 8, DRS is 4.22 for PTC versus 3.96 for sequential rollout.

• Qwen-Image-MTEdit is the strongest evaluated open-source model on the step-wise sub-bench. It reaches Turn 8 scores of IF 7.45, CC 6.42, and DRS 4.33, and holds the highest DRS among compared methods through Turn 8. Its base model Qwen-Image-Edit-Causal scores IF 3.17, CC 2.41, and DRS 4.06 at Turn 8.

• It still trails the strongest closed-source models on instruction following and content preservation. For example, Nano Banana Pro reports IF 8.83, CC 7.51, DRS 4.05 at Turn 8; the authors acknowledge this gap explicitly.

• Long chains expose failure modes in other methods. VINCIE collapses early (Turn 1 IF 1.13, CC 4.31, DRS 2.47 on the step-wise sub-bench; 0.01/0.02 by Turn 2). OmniGen2, the FLUX series, SenseNova-U1, Lance, and BAGEL degrade sharply by later turns — for instance OmniGen2 reaches 0.00/0.00 at Turn 8, and Lance at 0.00/0.00 at Turn 8 with DRS 4.11.

• Context-aware results are competitive but not uniformly best. On the persistent-constraint sub-bench, Qwen-Image-MTEdit reports 8.20 IF*, 8.32 CC*, and 4.27 DRS at Turn 4. On textual-reference it reports 8.24 / 8.12 / 4.17 at Turn 4, and on short-instruction 7.48 / 8.12 / 4.25 at Turn 4.

• A denser canvas layout was tested and rejected. Preliminary experiments with a 3×3 PTC layout caused noticeable degradation in per-panel quality and cross-turn consistency, so 2×2 became the default.

Methodology in Plain English

The core idea is to stop generating editing turns one after another and instead generate several turns at once inside a single image.

1. Pick good starting images. The team gathers candidate images from LAION, Open Images, and DenseFusion, keeping only those whose shorter side is at least 1024 pixels and whose aesthetic score from Gemini 3.1 Pro is at least 8 on a 0–10 scale.

2. Ask a VLM to write a "PTC prompt." Given a source image, a vision-language model plans an editing sequence and writes an instruction that tells a strong closed-source image model (such as GPT Image 2 or Nano Banana 2) how to draw a canvas containing multiple outcomes of that sequence.

3. Generate the packed trajectory canvas. The closed-source model produces one 4K square canvas with four panels in a 2×2 layout, so each panel is roughly 2K resolution. Because all panels are drawn in one shared visual context, the later panels do not inherit the accumulated artifacts they would in a chained rollout.

4. Turn the canvas into a training trajectory. Gemini 3.1 Pro discards failed canvases, then reads the source image plus the retained canvas to write user prompts for each turn. The canvas is split into panels, each panel becoming the target image for its turn. SIFT-based feature matching plus image warping then improves pixel-level alignment within each editing pair.

5. Extend to eight turns. Under the 2×2 layout one pass yields a four-turn sample; to get eight turns, the last generated image is used as a new source image and the pipeline is run again.

6. Train a model. Qwen-Image-Edit-Causal is fine-tuned with block causal attention (each turn sees earlier turns but not future ones) and 3D interleaved RoPE over an "[Image]-[Text]-[Image]-…" sequence, using teacher forcing so multiple turns are trained jointly. Training uses AdamW at a fixed learning rate of 5×10⁻⁵, weight decay 10⁻², betas (0.9, 0.999), 20,000 steps on 32 H100 GPUs with per-GPU batch size 1 in bfloat16.

7. Evaluate. MTEdit-Bench uses Qwen 3.5 Plus as a VLM judge scoring IF and CC from 0 to 10, with DRS computed as (VisualQuality-R1 + RALI)/2 on a 0–5 scale. If IF or CC falls below 6 at any turn, both are set to 0 for all later turns in that trajectory, reflecting the causal nature of multi-turn editing; DRS is never zeroed.

Why This Matters

Research impact. The paper argues against the field's default data recipe — chaining a strong model's outputs — by showing quantitatively that the recipe itself injects visual drift into training data, not just into inference. It also pushes benchmark design toward long (eight-turn) chains and toward a drift-specific metric, since the authors note many existing benchmarks use sequences of only around three turns and overlook visual-quality drift.

Real-world applications:

• Iterative creative design: a designer refining a layout, product shot, or illustration through many successive small instructions.
• E-commerce and advertising asset revision: updating a product image's background, then lighting, then props, while keeping the product itself unchanged across turns.
• Photo retouching workflows where a user gives a few broad instructions and then short follow-ups ("warmer," "a bit more") that depend on earlier context.
• Assistive or conversational image editing tools that must remember constraints stated several turns earlier, such as "keep the logo in the top-left."

Industry relevance. The dataset is generated with proprietary models (GPT Image 2, Nano Banana 2, Gemini 3.1 Pro, Seedream 5.0 Lite) and the paper notes that one PTC pass produces targets for multiple turns, reducing calls to expensive image-editing models. That makes the pipeline a practical template for teams building multi-turn editing training data without hand annotation. The released model is built on the open Qwen-Image-Edit-Causal line, so the result is directly usable by open-source practitioners.

Future Directions

• Pushing past eight turns. The paper's longest trajectories are eight turns, assembled by running the pipeline twice. Whether drift stays controlled at 16 or more turns, and how the 2×2 canvas bottleneck should be redesigned, is open.
• Denser or differently shaped canvases. The 3×3 experiment degraded panel quality and consistency, so a natural follow-up is making PTC generation work at higher panel counts — or layouts better suited to non-square editing targets — to get longer trajectories per generation pass.
• Closing the gap to closed-source models. Qwen-Image-MTEdit trails the strongest closed-source models on instruction following and content preservation, so improving instruction fidelity without sacrificing drift robustness remains an open problem.
• Better context-aware supervision and evaluation. Context-aware data is 44% of the dataset. Whether these capabilities can be pushed further — especially short-instruction and persistent-constraint editing, where scores vary notably across models — is left for future work, as is reducing reliance on VLM judges for scoring.

Target Audience

Researchers and engineers working on image generation and editing, particularly those building training datasets or fine-tuning instruction-based editing models. It is also relevant to benchmark designers interested in multi-turn evaluation and drift metrics, and to applied teams in creative tooling, e-commerce imaging, or any product where a user refines an image through a series of instructions. Readers without background in diffusion/flow-based editing models, VLM judges, and attention/positional-encoding details will find the methods sections dense.

Authors’ abstract

Multi-turn image editing aims to modify an image according to a sequence of instructions, making it better aligned with human-AI collaborative creation than single-turn editing. However, constructing high-quality multi-turn training data remains challenging, as conventional sequential rollout accumulates errors and introduces visual-quality drift across editing turns. To address this challenge, we introduce MTEdit, a multi-turn image editing dataset built through a pipeline centered on packed trajectory canvas (PTC) generation. By jointly generating multiple editing states within a unified visual context, this pipeline reduces visual-quality drift across turns. Covering both step-wise and context-aware editing scenarios, MTEdit comprises 500K multi-turn image editing trajectories, with up to 8 turns and an average of 5.38 turns per trajectory, making it one of the most extensive datasets for multi-turn image editing. We further introduce MTEdit-Bench, a benchmark designed to evaluate both scenarios by assessing models' ability to follow editing instructions, maintain content consistency, resist visual-quality drift, and effectively use historical context across turns. Using MTEdit, we fine-tune Qwen-Image-Edit-Causal to obtain Qwen-Image-MTEdit. On MTEdit-Bench, we evaluate a broad range of closed-source and open-source image editing models, including Qwen-Image-MTEdit. Experimental results show that Qwen-Image-MTEdit achieves the best performance among the evaluated open-source models, demonstrating the effectiveness of MTEdit in improving models' multi-turn image editing capabilities.