MT Forcing: Towards Stable and Consistent Multi-Turn Image Editing with Autoregressive Diffusion

AI summary

Overview

Research area: Computer Vision — instruction-based image editing with unified multimodal models (UMMs), specifically multi-turn editing with autoregressive diffusion.

Technical level: Advanced (assumes familiarity with diffusion distillation, distribution matching, and autoregressive sequence models).

Scope: A training paradigm that combines single-turn distribution-matching distillation with teacher-forced supervision to stop visual-quality and content drift across successive editing turns.

What This Paper Is About

Multi-turn image editing lets a user refine an image through a sequence of instructions, but existing editors drift after only a few turns: artifacts accumulate, fine detail degrades, regions outside the requested edit change unintentionally, and earlier edits are forgotten. The authors trace this to exposure bias — the model is trained on clean reference histories but at inference must condition on its own imperfect outputs. The paper's goal is a training recipe that keeps visual quality stable and content consistent across many turns.

Key Contributions

1. Single-Turn Distribution Matching Distillation (Single-Turn DMD): reorganizes all instructions up to the current turn into a "bundled prompt" expressing cumulative intent, and uses a frozen teacher to execute that intent directly on the source image in one turn. The student's autoregressive self-rollouts are aligned to this teacher distribution via a DMD loss, so supervision arrives without recursive error accumulation.
2. Teacher-Forcing Guidance: supervises history-dependent transitions on consistent reference trajectories using flow matching, applied as a warmup stage before distillation and as a lower-weight regularizer during distillation. This targets what per-turn distribution matching provably cannot: the model's dependence on the specific visual history.
3. MT Forcing: the combined objective (DMD plus weighted teacher-forcing loss) that yields more stable visual quality, stronger cross-turn consistency, and better instruction following.
4. A new metric, Pixel: built on MatchAnything-RoMa, it measures mean matched-point displacement in pixels between successive editing outputs, quantifying geometric alignment (lower is better). The paper also adopts MTEdit-Bench's IF, CC, and DRS metrics.

Main Findings

• Later turns improve substantially: on the eight-turn step-wise sub-bench of MTEdit-Bench, MT Forcing outperforms both base editors on all four metrics from Turn 4 onward, with gains growing at later turns.
• Turn-8 gains are large: IF and CC improve by 7.45 and 6.82 on SenseNova-U1, and by 4.65 and 4.46 on SenseNova-U1.5.
• Drift robustness is held rather than degraded: on SenseNova-U1.5, MT Forcing keeps DRS between 4.20 and 4.35 throughout the session, while the baseline falls from 4.09 to 3.03.
• Geometry is much better preserved: eight-turn mean Pixel on SenseNova-U1.5 drops from 6.34 to 0.74, an 88.4% reduction in matched-point displacement.
• Both components are necessary, and complementary: ablation eight-turn means on SenseNova-U1.5 — baseline 8.47 IF / 7.09 CC / 3.83 DRS / 6.34 Pixel; DMD only 9.02 / 7.64 / 4.12 / 2.51; Teacher-Forcing Guidance only 9.10 / 7.83 / 3.92 / 2.74; full MT Forcing 9.66 / 8.40 / 4.30 / 0.74.
• The two components fix different failures: DMD alone reduces exaggerated textures but leaves content inconsistent (wood-grain patterns change between Turns 4 and 8); guidance alone improves consistency but visual degradation persists under self-generated histories.
• There is a consistency/sharpness trade-off in the regularization weight: λ_TF of 1 and 0.5 visibly blur fur detail, while 0.25 and 0.1 restore finer strands and sharper coat boundaries.
• Early turns are not uniformly better: on SenseNova-U1.5 at Turn 1, IF moves from 9.90 to 9.74 and CC from 9.49 to 8.82, i.e. some small regressions before the method's advantage appears.

Methodology in Plain English

The setup: a session has a source image plus a sequence of instructions. At each turn the model sees the source, all previous instructions, and — critically — its own previously generated images, cached in a key-value memory. Training instead typically shows it perfect reference histories, and that mismatch is what makes errors snowball.

The authors' fix has two halves.

Half one — grade each turn against a one-shot teacher. A single editing turn usually causes little damage, and an editing model can perform several actions at once. So for turn i, the researchers merge instructions 1 through i into one "bundled prompt" (e.g., "add a vase" then "make it blue" becomes "add a blue vase") and have a frozen teacher apply that combined edit to the original source image in a single pass. The student, meanwhile, edits its own imperfect chain of outputs. A distribution-matching loss then pulls the student's per-turn output distribution toward the teacher's — the teacher's result is clean because it never touched the student's degraded history. The teacher sees weaker conditioning (just the source plus bundled prompt) than the student (full history), and the paper shows this one-way mapping is still enough to define a valid learning target through the student's rollout marginal.

Half two — anchor behaviour to reference histories. Matching per-turn distributions is provably insufficient: a student that ignored history entirely and sampled independently from the teacher at every turn would achieve a perfect DMD loss while still changing content between turns (the paper illustrates this with a teapot whose floral pattern shifts). So the model is also trained, with standard flow matching, to reproduce the target image at each turn given a coherent reference history. This is done first as a warmup before distillation, then kept at a lower weight during distillation to stop the distribution-matching updates from erasing history-dependent behaviour.

Efficiency detail: gradients are truncated across turns — historical KV embeddings are detached and gradients flow only through the sampled denoising step of the current turn. A separate online "fake" model estimates the student's score, following DMD2 practice.

Why This Matters

Impact on research. The paper gives a concrete diagnosis of multi-turn failure (exposure bias) and a decomposition of the remedy into two objectives with distinct jobs — one for per-turn quality, one for history dependence — plus an argument for why the first alone cannot work. It also shows that unlike video, multi-turn editing has no bidirectional teacher that can jointly generate a whole editing trajectory, so the standard autoregressive-distillation playbook does not transfer directly. The Pixel metric adds a quantitative handle on geometric preservation that the existing IF/CC/DRS triad does not capture.

Real-world applications:

• Interactive photo-retouching tools where a user issues dozens of incremental requests in one session.
• Marketing and advertising creative iteration, where a product shot is progressively restyled without losing the product.
• E-commerce and catalog imagery, where identity of the item must survive repeated edits.
• Storyboarding and pre-visualization, where a scene is refined shot by shot and earlier decisions must persist.

Industry relevance. The method is a training-time add-on applied to two existing editors (SenseNova-U1 and SenseNova-U1.5) rather than a new architecture, which makes it a plausible drop-in for teams already shipping autoregressive UMM editors. The measured 88.4% reduction in matched-point displacement and the flat DRS curve map directly onto the user-visible complaint that an editor "ruins the picture after a few rounds."

Future Directions

• What teacher is ideal? The paper notes the absence of a bidirectional UMM that jointly generates full editing trajectories — constructing one, or a better surrogate, is an open problem.
• Tuning the consistency/quality balance. The λ_TF sweep (1, 0.5, 0.25, 0.1) shows trade-offs between blur and detail; a principled or adaptive schedule is not reported.
• Earlier-turn behaviour. Since SenseNova-U1.5 shows small IF/CC dips at Turn 1 and Turn 2, understanding and removing these regressions is a natural next step.
• Generalization beyond the evaluated setting. All results are on two SenseNova-U1 variants and the 100-session, eight-instruction step-wise sub-bench of MTEdit-Bench; whether the paradigm holds for other UMMs, longer sessions, or different instruction styles is not reported.

Target Audience

Researchers and engineers working on image editing, unified multimodal models, and autoregressive or few-step diffusion — particularly those who already understand distillation objectives such as DMD/DMD2 and teacher forcing, and who need multi-turn stability rather than single-image quality. Practitioners building interactive editing products will find the empirical drift analysis and the Pixel metric useful even without following the full derivation.

Authors’ abstract

Multi-turn image editing aims to modify an image according to a sequence of instructions, enabling more effective interactive visual content creation for users. However, existing methods often exhibit noticeable drift after only a few editing turns, manifesting in two aspects: 1) unstable visual quality, with accumulating artifacts and degraded details; and 2) inconsistent image content, with unintended changes to preserved regions and prior edits. To address these challenges, we propose MT Forcing, a novel training paradigm for multi-turn image editing with unified multimodal models (UMMs) based on autoregressive diffusion. Specifically, we propose Single-Turn Distribution Matching Distillation (Single-Turn DMD), which supervises student self-rollouts with a teacher that applies cumulative edits to the source image in a single turn, providing visual-quality guidance without recursive error accumulation. Additionally, we introduce Teacher-Forcing Guidance, which supervises history-dependent transitions on reference trajectories during warmup and as a regularizer during distillation, encouraging cross-turn content consistency beyond what per-turn distribution matching alone can ensure. Extensive experiments demonstrate that our approach significantly improves the multi-turn image editing capabilities of UMMs, yielding more stable visual quality and stronger cross-turn consistency.