ACEdit: Stable Multi-Turn Image Editing via Asymmetric Conditioning

AI summary

Overview

Research area: Computer Vision — generative image editing, specifically multi-turn (iterative) text-guided image editing with diffusion/flow-matching models.

Technical level: Advanced. The method builds on score distillation (SDS/VSD/DMD), flow-matching velocity-to-score conversion, and LoRA fine-tuning; the paper assumes familiarity with these.

Scope: The paper proposes a training scheme — "asymmetric conditioning" — that keeps a multi-turn image editor stable when it is repeatedly applied to its own outputs, and evaluates it on a new ten-turn benchmark (EdiVal-10) against training-free drift-reduction methods and a trained multi-turn editor (MT-OPSD).

What This Paper Is About

Image editors now produce strong single edits, but when a user chains edits — each output becoming the next input — small distribution shifts accumulate and the image degrades: fine detail is lost, colors drift, and content changes that nobody asked for. The paper's goal is to train the editor so that it stays stable across long self-generated editing sequences, without adding any cost at inference time. The core idea is to force the model to train on its own imperfect intermediate outputs, and to supervise those outputs with a teacher that always looks back at the original clean image plus a single combined instruction.

Key Contributions

1. Self-Forcing applied to image editing. The editor (student) is trained on rollouts — sequences where each turn's output becomes the next turn's input — so that training matches the distribution it actually meets at inference. During training, rollouts are chains of E = 10 edits.

2. Asymmetric conditioning between student and teacher. The student sees its own previous output plus the current instruction; the frozen teacher sees the original clean image I₀ plus a compound instruction h_e = (c₁, …, c_e) aggregating all edits up to that turn. This prevents the teacher's supervision signal from inheriting the student's accumulated errors.

3. A distillation objective for self-generated trajectories. Because ground-truth images cannot serve as paired targets once a rollout diverges, supervision comes from score distillation: the difference between the frozen teacher's velocity prediction and a trainable auxiliary editor's prediction (both conditioned identically on I₀ and h_e) provides the gradient to the student, following the VSD/DMD teacher–student formulation.

4. EdiVal-10, a ten-turn evaluation benchmark, plus a quantitative, qualitative and ablation study. The authors argue existing benchmarks are too short for drift to build up (EdiVal chains three edits, MagicBrush at most three, MSE-Bench five, Omni IIE Bench averages 4.35 turns with only 9 of its 260 reaching ten).

Main Findings

• Higher final-image quality after ten edits. On EdiVal-10 (252 chains), the model scores 3.19 on the KRIS-Bench image-quality score (1–5, VLM-judged), significantly above every other method; the smallest margin is over Emu Edit at 0.30 ± 0.08. For reference, the FLUX.2-klein-9B base model scores 2.73, VAE-LFA 2.80, FreqEdit 2.83, Emu Edit 2.90, MT-OPSD 1.95, and the undistilled base model 1.42.

• Fewest unintended changes. Whole-image KRIS consistency is 3.85, versus 3.04–3.32 for the training-free methods, 3.03 for the base model and 1.29 for MT-OPSD. Background LPIPS distance is 0.254, versus 0.351 for the base model.

• Anchor objects are preserved competitively. DINOv2 distance on unedited anchor objects is 0.203, statistically indistinguishable from Emu Edit's 0.196 (difference −0.007 ± 0.012), while all other methods are significantly worse. Emu Edit's only close margins are on these two local metrics (background 0.264, +0.010 ± 0.005, t = 2.1), not on whole-image consistency.

• Instruction following is not sacrificed. KRIS instruction-following at turn 10 is 3.38, not significantly different from the highest score (FreqEdit, 3.39) and significantly above MT-OPSD (3.16) and its base model (3.10) — evidence that stability does not come from silently skipping edits.

• Single-edit quality is better preserved. Applying each method once to each of the 606 English instructions of GEdit-Bench, HPSv3 drops by 0.18 for this model versus 0.41 for the base model (difference 0.23 ± 0.02); every other method loses significantly more.

• The teacher's conditioning matters most (ablation). Letting teacher and auxiliary editor also see the previous student output drops consistency from 3.85 to 2.81; conditioning them only on the previous output and current instruction drops it to 2.28 — below the 3.03 of the untouched pretrained base model. Conditioning score models on intermediate student outputs appears to reinforce artifacts rather than correct them.

• Giving the student the source image backfires. Adding I₀ as an extra student reference raises consistency to 4.08 but collapses instruction following from 3.38 to 2.80; qualitatively, earlier edits are undone (a stool recolored yellow at turn 3 is blue again by turn 6).

• Rollout length during training matters. Training on 5-edit rollouts instead of 10 still beats the base model after ten turns (consistency 3.29 vs 3.03) but recovers only about a third of the full gain (3.85), and is significantly worse on every preservation measure.

• Cheap to train, free at inference. LoRA of rank 16 with 27.8M trainable parameters (0.3% of the model), 505 iterations (5 warm-up + 500, of which 100 student updates), about 4.4 GPU-hours on a single NVIDIA B200, and no inference-time overhead.

• Limitations inherit from the backbone. In a nine-edit sequence, both the base model and this model follow "Make the hiker smile" but also remove sunglasses added in an earlier edit. The method reduces drift; it does not exceed the base model's editing competence.

Methodology in Plain English

The problem in one picture. An editor is trained on clean input–target pairs. At inference it is fed its own slightly imperfect output, then its own slightly-more-imperfect output, and so on. Errors compound. The fix, borrowed from autoregressive video generation ("Self-Forcing"), is to simply train it in the situation it will be in — on its own outputs.

Why that is hard. Once a rollout is generated by the student, its intermediate images no longer correspond to any ground-truth image. Edit outcomes are not unique, so paired supervision is meaningless beyond the first turn.

The solution: distill from a teacher that never sees the mess. A frozen, undistilled editor (FLUX.2-klein-base-9B) acts as the teacher. Crucially, it is not shown the student's corrupted intermediate image. Instead it is shown the original clean image plus a merged instruction — "add handbag" → "make it red" → "make it leather" → "add a gold buckle" becomes a single "add a red leather handbag with a gold buckle". The teacher scores whether the student's proposed result is what that compound edit should look like, so the training signal reflects the intent of the edits without inheriting the drift along the way.

The mechanics. The student (FLUX.2-klein-9B, a 4-step distilled model) runs a 10-turn rollout in latent space — outputs pass between turns without VAE decoding/encoding, and gradients are not backpropagated across turns. At a randomly chosen turn and denoising step, the student's clean-image estimate is noised and evaluated by two models: the frozen teacher and a small auxiliary editor trained alongside the student to model the distribution the student actually produces. The difference between the two predictions is the distillation gradient. The auxiliary editor is trained with a standard flow-matching loss on the student's own clean estimates, and the two are updated alternately (auxiliary every iteration, student every fifth).

How it was tested. 96 synthetic 1024×1024 images generated with FLUX.2-klein-9B, each paired with a chain of 10 local edit instructions (960 instructions total), all local — add, remove, replace, resize, or attribute changes. Evaluation uses EdiVal-10: 252 chains of 10 cumulative edits on real photographs from GEdit-Bench, with grounded objects, eight local edit types, and one or two reserved "anchor" objects no instruction may touch, so unrequested change is measurable. Metrics combine a VLM judge (Gemma-4-31B) using KRIS-Bench prompts, ΔHPSv3, DINOv2 anchor distance, and LPIPS background distance.

Why This Matters

For research. The paper makes a clean, transferable point: for iterative generative systems, stability should be learned over the whole recursive process, not at individual steps — and the supervision source for that process must be anchored to a clean reference, or it will simply launder the model's own errors back into it. The ablation showing that a teacher conditioned on the student's previous output scores below the untouched base model (2.28 vs 3.03 consistency) is a concrete, reusable warning. EdiVal-10 also fills a measurement gap: the authors document that the most-cited multi-turn benchmarks chain only three to five edits, and even the 16-turn benchmark averages 4.35.

Real-world applications:

• Consumer photo editing and creative tools, where users iteratively refine an image through many small instructions in a session.
• E-commerce and product photography, where a catalog image is progressively modified (background, color, props) and must not drift away from the actual product.
• Marketing and design iteration, where a concept image is refined across many rounds without accumulating color casts or losing texture.
• Assistive or agentic editing pipelines, where a planning agent issues long sequences of edits automatically and cannot rely on a human to notice silent degradation.

Industry relevance. The training cost is small (4.4 GPU-hours on one B200, 0.3% of parameters via LoRA) and there is zero inference overhead — the deployed 4-step sampler is unchanged. That makes it a plausible post-training step for existing editing backbones rather than a new architecture.

Future Directions

The paper's Conclusions/Limitations/Future Work section is truncated in the provided content, so its stated future-work agenda is not fully available; only the limitation that the method inherits the base model's editing capabilities and failure modes is reported.

Logical next steps the work raises:

• Beyond local edits. Every training and evaluation edit in EdiVal-10 is local. Whether the same anchoring holds for global edits (style, weather, color grading) — which the authors explicitly criticize LME-Bench for emphasizing — is not reported.
• Fixing inherited failures. The sunglasses-removal case shows the method stabilizes drift but does not repair backbone-level instruction errors; combining this objective with trajectory-level rewards or stronger instruction supervision is an open question.
• Longer than ten turns. Training rollouts were 10 edits, matching evaluation; the 5-edit ablation shows shorter rollouts recover only about a third of the gain. How far the benefit extends past ten turns is not reported.
• Cheaper or better teachers. The teacher is a frozen undistilled model of the same family, evaluated at every turn; whether weaker, cheaper, or cross-family teachers suffice, and how the compound-instruction formulation degrades as instruction chains grow, are untested here.

Target Audience

Researchers and engineers working on generative image editing, diffusion/flow-matching post-training, and distillation (SDS/VSD/DMD); practitioners deploying iterative editing features who need stability without inference cost; and benchmark designers interested in how ten-turn evaluation was constructed for EdiVal-10. Readers without background in score distillation will find the method sections demanding, though the problem framing and results tables are accessible.

Authors’ abstract

Image editing models produce impressive results for individual edits, but their quality often deteriorates over multiple editing turns. Each editing turn introduces small distribution shifts that accumulate over successive edits, creating a gap between the input distributions encountered during the editor's original training and multi-turn inference. We introduce a training scheme that makes multi-turn image editing stable under this accumulating distribution shift. A key is to force the editor to operate on its own outputs from previous editing turns during training, bringing the principle of Self-Forcing to image editing. The main challenge is that self-generated editing trajectories deviate from ground-truth trajectories, making direct supervision unreliable. We address this challenge through a distillation objective that leverages a teacher image editor to provide a reliable training signal when the model operates on its own previous outputs. We introduce an asymmetric conditioning scheme: while the student performs edits sequentially on its own outputs, the teacher remains anchored to the original clean image and receives a compound editing instruction. Experiments show substantially reduced degradation over long multi-turn sequences while maintaining editing quality and instruction adherence.