On-Policy Self-Distillation for Multi-Turn Image Editing

AI summary

Overview

Research area: Computer vision, specifically instruction-based image editing with diffusion/flow-matching generative models, plus on-policy self-distillation and multi-turn robustness evaluation.

Technical level: Advanced. The paper assumes familiarity with flow matching, velocity-field parameterization, classifier-free guidance, LoRA adaptation, and distillation objectives, though the central problem (error accumulation across repeated edits) is stated in accessible terms.

Scope: The paper diagnoses multi-turn editing collapse in instruction-based editors, attributes it to a train–test mismatch in the conditioning distribution, and proposes MT-OPSD, an on-policy self-distillation framework, along with LME-Bench, a 100-session ten-turn evaluation benchmark.

What This Paper Is About

Instruction-based image editors work well when an instruction is applied to a clean source image, but degrade when a user applies edits repeatedly, each turn conditioning on the model's own previous output. The authors attribute this to a train–test mismatch: models are trained on clean source images but at inference must condition on their own imperfect outputs, and these small model-induced errors accumulate. Their goal is to make an editor robust over long editing sequences without needing multi-turn annotations, ground-truth edited images, or an external teacher.

Key Contributions

1. Diagnosis of multi-turn collapse. The authors demonstrate that recursive editing degrades across modern image editing models and attribute it to a train–test mismatch in the conditioning distribution, drawing an analogy to exposure bias in autoregressive video generation. A diagnostic in which the model is repeatedly instructed to leave the image unchanged exposes measurable model-induced drift at each turn.

2. MT-OPSD. An on-policy self-distillation framework that transfers the model's own clean-conditioned editing behavior to self-generated states. It requires no multi-turn annotations and no external teacher, using the clean source image as privileged teacher context and self-generated rollout states as student context.

3. LME-Bench. A benchmark of 100 ten-turn editing sessions (1,000 editing instructions) covering both local and global operations, evaluated turn-by-turn for editing accuracy, consistency with the previous state, and image quality.

4. Empirical validation across three backbones. Experiments on LME-Bench, MSE-Bench, and ImgEdit show improved long-horizon success and reduced collapse while largely preserving single-turn editing quality.

Main Findings

• Multi-turn collapse is backbone-agnostic. On LME-Bench, the three open-source base models drop to SR@10 of 0.03–0.15 and rise to CR@10 of 0.25–0.61, indicating that degradation is a limitation of the standard single-turn training paradigm rather than a model-specific failure.

• MT-OPSD substantially improves long-horizon success. Across all three backbones it raises SR@10 to 0.38–0.52 and reduces CR@10 to at most 0.04, with no loss of early-turn performance. On Qwen-Image-Edit-2511, SR@10 goes from 0.03 to 0.44 and CR@10 from 0.55 to 0.02; on FireRed-Image-Edit from SR@10 0.15 to 0.52 and CR@10 0.61 to 0.03; on FLUX.2-klein-base from SR@10 0.12 to 0.38 and CR@10 0.25 to 0.04.

• Gains transfer beyond training rollout depths. The rollout curriculum typically saturates at around four turns, yet the improvements persist through ten-turn evaluation on LME-Bench.

• Training-free baselines give model-dependent gains. Emu Edit notably improves Qwen-Image-Edit-2511, VAE-LFA is more effective on FireRed-Image-Edit, and FreqEdit generally degrades long-horizon performance.

• Proprietary systems span a wide range. GPT-Image-1 also degrades severely and is outperformed by MT-OPSD across all reported turns and metrics on LME-Bench. Nano Banana rarely collapses (CR 0.00 at all reported turns) but has lower SR (0.67/0.66/0.63/0.56), attributed primarily to failures on global edits that occur predominantly in earlier turns. GPT-Image-2 (SR 1.00/0.98/0.98/0.96, CR 0.00) and Nano Banana Pro (SR 1.00/1.00/0.97/0.91, CR 0.00) remain stronger at long horizons.

• Consistent improvement on MSE-Bench. MT-OPSD improves SR@5 from 0.24 to 0.51 on Qwen-Image-Edit-2511 and from 0.40 to 0.69 on FireRed-Image-Edit, while matching FLUX.2-klein-base at 0.49. Emu Edit remains competitive in this local-edit-dominated setting.

• Single-turn quality preserved. On ImgEdit, MT-OPSD shows only minor score changes across all three models (for example, 4.49 versus 4.51 for Qwen-Image-Edit-2511, and 4.52 versus 4.56 for FireRed-Image-Edit).

• Both branches are necessary but play different roles. Removing the editing branch causes SR@10 to fall to 0.00 and CR@10 to rise to 0.80. Removing the identity branch retains high SR@10 (0.49, slightly above the full method) but makes edits less controlled: CR@10 rises from 0.02 to 0.06, MSE-Bench SR@1 drops from 0.98 to 0.85, and ImgEdit falls from 4.49 to 3.88; qualitative results show this variant over-edits untargeted content across turns.

• On-policy supervision matters more than pseudo-target flow matching. Replacing on-policy velocity matching with SFT-style flow-matching supervision on teacher-generated targets reduces SR@5 from 0.91 to 0.59 and SR@10 from 0.44 to 0.03, while CR@10 rises from 0.02 to 0.67, surpassing the base model.

• Curriculum and promotion each contribute. Fixing rollout depth at four turns reduces SR@10 from 0.44 to 0.17 and increases CR@10 from 0.02 to 0.19. Keeping the pretrained initialization as a fixed teacher maintains CR@10 at 0.06 but limits SR@10 to 0.19, while achieving the highest ImgEdit score among ablations at 4.55.

Methodology in Plain English

The authors build on the observation that a pretrained editor already edits well when conditioned on a clean image. The problem is not that the model cannot edit; it is that after a few turns it is conditioning on images containing its own accumulated errors.

Their approach, MT-OPSD, first constructs "self-generated rollout states." Starting from a clean source image, the current student model is repeatedly asked to reproduce the image unchanged using an identity instruction such as "Make everything unchanged." Because the intended content does not change, any difference between the rollout state and the clean source mainly reflects the model's own accumulated errors, which isolates that error component from the intended semantic changes earlier edits would introduce.

Training then alternates between two branches. The identity branch supervises the model on the rollout state using that state itself as the target — not the clean source — which prevents further drift rather than attempting to restore the original. The authors report that using the clean source as target was difficult to optimize because it requires correcting errors accumulated over multiple turns in a single step. The editing branch pairs the same rollout state with a real editing instruction and matches the student's velocity to that of a teacher conditioned on the clean source, following the sparse query-based velocity matching of DanceOPD. The student runs an N-step denoising process, a small number of query steps are sampled, and teacher and student velocities are matched at the same stop-gradient student states with the same classifier-free guidance scale.

Two mechanisms control how training evolves. An adaptive rollout curriculum increases rollout depth by one turn only when measured pixel drift stays below a threshold for a fixed number of consecutive training steps, avoiding the instability of starting at a large depth. Gated teacher promotion lets improved student checkpoints replace the teacher, with candidates evaluated asynchronously by a VLM judge on a held-out gate set and promoted only when a long-horizon success/collapse criterion is met; the judge is used only for checkpoint selection, not as a training target or reward signal.

Implementation details reported: LoRA rank 32 with alpha 64, AdamW, learning rates of 3×10⁻⁴ for Qwen-Image-Edit-2511 and FireRed-Image-Edit and 1×10⁻⁴ for FLUX.2-klein-base, constant schedules, 2,000 training samples, 512² training resolution, identity rollout sampling steps of 30/30/50, guidance scale 4.0, an editing-branch trajectory of 10 steps with 2 query steps per update, query distributions Beta(5,5) for two backbones and Beta(2,5) for FLUX.2-klein-base, identity loss weight 10, initial rollout depth 2, maximum rollout depth K_max = 10, drift thresholds of 9.0/10.0/10.0, and patience 10. Training uses roughly 2,000 source-image/instruction pairs from OmniEdit without the corresponding ground-truth edited images; each run uses four NVIDIA H100/H200 GPUs for training plus another four GPUs for asynchronous gate evaluation. The editing and identity branches are sampled at a ratio of 2:1.

Why This Matters

Impact on research. The paper reframes multi-turn editing degradation as a conditioning-distribution mismatch analogous to exposure bias, and shows that a model's own clean-conditioned behavior can serve as supervision without an external teacher, paired target images, or multi-turn annotations. It also introduces a long-horizon benchmark where none previously extended beyond five turns, and it provides an ablation showing which components carry the improvement.

Real-world applications:

• Iterative photo editing workflows where a user refines an image over many rounds (lighting, color palette, object repositioning, stylistic transformation), each edit applied to the previous result.
• Creative and design tooling where a session accumulates dozens of small adjustments and identity drift would be unacceptable.
• Product or portrait retouching pipelines that must preserve subject identity and structure across repeated edits.
• Any deployment of open-source editors as interactive assistants, where users do not reset to a clean image between requests.

Industry relevance. The method is a LoRA adaptation applied on top of existing backbones, so it does not require training a new model from scratch. The reported results show that open-source editors such as Qwen-Image-Edit-2511, FireRed-Image-Edit, and FLUX.2-klein-base can close part of the long-horizon gap with proprietary systems like GPT-Image-2 and Nano Banana Pro. Because the framework requires only roughly 2,000 source-image/instruction pairs and no multi-turn annotations, the data cost of adopting it is comparatively low.

Future Directions

• Extending beyond the saturation depth. The rollout curriculum typically saturates at around four turns while gains persist to ten, so the relationship between training rollout depth and long-horizon robustness remains an open question.
• Improving the long-horizon ceiling. The strongest proprietary systems still lead at late turns on LME-Bench, leaving room to close that gap further.
• Comparing against training-based multi-turn methods. The paper notes VINCIE is the only training-based multi-turn method with public code and checkpoints, and reports its comparison separately in Appendix E, which is not included in the available content; a fuller head-to-head comparison remains to be established.
• Reducing reliance on external judging. Gated teacher promotion depends on a VLM judge over a 23-session gate set. Whether promotion can be driven by cheaper, self-contained signals is not addressed.

Target Audience

Researchers and engineers working on generative image editing, diffusion and flow-matching models, or distillation-based post-training. Practitioners building interactive or iterative editing products will benefit from the diagnosis and the benchmark, while those focused on evaluation methodology will find LME-Bench's turn-by-turn success and collapse metrics useful. The paper's framing is accessible to graduate students entering the area, but reproducing the method requires working knowledge of flow matching and velocity matching.

Note: several referenced appendices (the full edit taxonomy and composition rules in D.1, the complete judging prompts and thresholds in D.2, the full promotion rule in C, the VINCIE comparison in E, and the qualitative results in F and G) are referenced but not included in the available paper content, so their specific figures are not reported here.

Authors’ abstract

Instruction-based image editing has achieved strong performance in single-turn settings, yet practical editing is often iterative, with each instruction applied to the output of the previous turn. We find that existing editing models degrade rapidly under recursive editing and attribute this failure to a train-test mismatch in the conditioning distribution: models are trained on clean source images but must repeatedly condition on their own imperfect outputs at inference time. To address this, we propose MT-OPSD, an on-policy self-distillation framework that trains the model on self-generated conditioning states with editing supervision from a clean-conditioned teacher, without requiring multi-turn annotations. We further introduce LME-Bench, a benchmark of 100 ten-turn editing sessions for evaluating long-horizon robustness. Experiments across three editing backbones show that MT-OPSD substantially improves long-horizon editing success and reduces multi-turn collapse while largely preserving single-turn editing quality.