Reproducing DragDiffusion: Interactive Point-Based Editing with Diffusion Models

AI summary

Overview

Research area: Computer Vision / Generative AI — specifically interactive, point-based image editing with diffusion models, presented as a reproducibility study (TMLR Reproducibility Challenge entry, arXiv:2602.12393v1 [cs.CV], 12 Feb 2026).

Technical level: Intermediate. A reader needs basic familiarity with diffusion models, LoRA, and latent-space optimization, but the paper is organized around clearly stated ablation claims and metric tables.

Scope: The paper independently re-runs the core quantitative ablations of DragDiffusion (Shi et al., 2024) using the authors' released code and the DragBench benchmark, tests their sensitivity to key hyperparameters, and adds one extension (multi-timestep latent optimization) that is not part of the original paper.

What This Paper Is About

DragDiffusion lets a user edit an image by dragging handle points to target locations, claiming that precise spatial control comes from optimizing a single diffusion latent at one intermediate timestep, plus identity-preserving LoRA fine-tuning and spatial mask regularization. The problem this paper addresses is whether those claims actually hold up when someone else runs the released code — and how fragile the results are to small changes in configuration. The goal is not to beat DragDiffusion, but to verify which of its design decisions are reproducible, which are sensitive, and whether a more expensive multi-timestep variant would do any better.

Key Contributions

1. A systematic reproduction of all five core quantitative claims of DragDiffusion — timestep selection, necessity of LoRA fine-tuning, extent of LoRA fine-tuning, mask regularization strength, and UNet feature-supervision level — each explicitly cross-referenced to the corresponding figure or table in the original paper.
2. A sensitivity analysis identifying that only a small number of hyperparameters (the optimized timestep and the motion-supervision feature level) are critically fragile, while others (LoRA duration, mask regularization strength) tolerate broader operating ranges.
3. A controlled extension testing multi-timestep latent optimization (jointly optimizing latents at timesteps {30, 35, 40}), which does not correspond to any reported result in the original paper.
4. A transparent account of reproduction ambiguities and implementation choices — DDIM inversion settings, random seed control, numerical precision, and GPU determinism — plus honest reporting of what was easy versus difficult to reproduce.

Main Findings

• Intermediate timestep wins (Claim 1). Optimizing at t = 35 gives the lowest Mean Distance (MD 34.90) versus t = 20 (MD 40.90) and t = 50 (MD 49.30). Image Fidelity is highest at t = 20 (0.8911), drops to 0.8466 at t = 35, and falls to 0.7666 at t = 50. The paper notes this matches the original's identification of roughly t = 30–40 as optimal.

• LoRA fine-tuning is essential (Claim 2). Without LoRA, MD is 55.68 and IF is 0.8646. With LoRA, MD falls to 34.90 and IF rises to 0.8822 — a relative MD improvement of approximately 37 percent.

• Diminishing returns on LoRA steps (Claim 3). Across {0, 20, 40, 80, 100, 120} steps, MD falls from 54.6062 (0 steps) to 41.0366 (20), 35.9723 (40), 34.8997 (80), reaches its best value of 34.5541 at 100 steps, then degrades to 35.5443 at 120. Image Fidelity peaks at 100 steps (0.8820). The default in the original implementation is 80 steps; the paper reports gains beyond 80 steps are modest.

• Moderate mask regularization is best (Claim 4). With λ = 0.0, MD is 36.9804 and IF drops to 0.7640 (visible background distortion). The original setting λ = 0.1 achieves the lowest MD (34.1272) with IF 0.8736. Stronger regularization, λ = 0.5 (MD 34.8613, IF 0.8771) and λ = 1.0 (MD 36.6451, IF 0.8823), raises fidelity but slightly worsens spatial accuracy.

• Mid-level UNet features win (Claim 5). Decoder Block 3 gives the lowest MD (35.1043, IF 0.8734). Block 1 performs poorly (MD 54.4055), Block 2 is better (MD 36.1107), and Block 4 yields the highest IF (0.9123) but much worse MD (46.1320). The reproduced ordering of decoder blocks exactly matches the original findings.

• Multi-timestep optimization does not help (extension). Compared with the single timestep baseline (MD 35.10, IF 0.8734, runtime 1.0×), the {30, 35, 40} multi-timestep variant has MD 36.28, IF 0.8719, and runtime 2.7×. The paper attributes this to redundancy among neighboring diffusion states, which are highly correlated in the mid-range of the diffusion process.

• Where reproduction was easy versus sensitive. DDIM inversion, single-timestep latent optimization, motion supervision, and the overall trends were reproducible with minimal modification. Performance proved sensitive to the optimized timestep, mask regularization strength, and the feature level used for motion supervision; LoRA fine-tuning also added non-trivial compute overhead and required careful checkpoint management.

• Compute footprint and setup. All experiments use Stable Diffusion v1.5, with LoRA rank 16, learning rate 5 × 10⁻⁴, up to 120 optimization steps, t = 35 by default, and DDIM steps fixed to 50. Hardware was an NVIDIA A100 40GB with an AMD EPYC 7453 28-Core CPU, 64 GB RAM, Ubuntu 22.04.3 LTS, Python 3.8.5, CUDA 11.7 (cudatoolkit 11.7.0), PyTorch 2.0.0, diffusers 0.24.0, transformers 4.27.0. Runtime was approximately 7 seconds per sample, approximately 1.8 minutes LoRA training per sample (80 fine-tuning steps), approximately 7.5 hours total evaluation, and an estimated compute budget of approximately 9 GPU-hours.

Methodology in Plain English

The authors took DragDiffusion's released codebase and DragBench benchmark and re-ran the paper's own experiments rather than building a new method. For each claim, they changed exactly one thing and held everything else fixed: the optimized timestep was varied across {20, 35, 50}; LoRA was turned on and off and then varied across {0, 20, 40, 80, 100, 120} training steps; the mask regularization weight λ was varied across {0.0, 0.1, 0.5, 1.0}; and motion supervision was applied to UNet decoder blocks {1, 2, 3, 4}. Every configuration was evaluated with the same DragBench subset, images, prompts, drag instructions, and random seed, using the two metrics from the DragBench protocol: Mean Distance (average Euclidean distance between target points and their final semantic locations, following the definition from DragGAN) and Image Fidelity (1 − LPIPS, where higher is better).

They then added one experiment the original paper did not run: instead of optimizing a single latent at t = 35, they optimized a set of latents at timesteps {30, 35, 40}, applying motion supervision and mask regularization independently at each timestep and accumulating gradients to update all selected latents, then compared accuracy, fidelity, and runtime.

Because full numerical determinism is not guaranteed on GPUs (non-deterministic CUDA and cuDNN kernels), the authors used fixed seeds and identical execution order across runs but compared results by relative trends rather than exact numerical equality. They retained the released code's default mixed-precision settings, noting that forcing full fp32 did not improve spatial accuracy and increased runtime.

Why This Matters

Research impact. Reproducibility studies are how a field separates real, robust effects from artifacts of one lab's setup. This paper shows DragDiffusion's five central claims hold up under independent re-execution, but also localizes the fragility: timestep selection and feature-supervision level are the load-bearing decisions, while LoRA duration and mask regularization strength have forgiving ranges once reasonable values are picked. That is actionable information for anyone building on the method, and the finding that multi-timestep optimization buys accuracy nothing while costing 2.7× runtime closes off an intuitive but unproductive direction.

Real-world applications (these follow from interactive point-based editing, the capability the paper evaluates):

• Photo retouching and object repositioning, where a user drags an object to a new spot while keeping the surrounding scene intact.
• E-commerce product imagery, adjusting the position or pose of a product in a catalogue shot without re-shooting.
• Creative design and concept art workflows, where drafters prototype layout changes by direct manipulation rather than text prompts.
• Film and animation post-production, making small spatial corrections to frames where text prompts cannot express exact geometric intent.

Industry relevance. The paper quantifies the practical cost of the method — approximately 7 seconds per sample, approximately 1.8 minutes of LoRA training per sample, approximately 9 GPU-hours for the full study — which is the kind of number a team needs before committing to a production pipeline. Its documentation of library-version friction and the need for careful checkpoint management is also directly useful to engineers integrating the released code.

Future Directions

• Extending beyond the tested backbone. The study is limited to Stable Diffusion v1.5; whether the same sensitivities hold for other diffusion backbones is not reported.
• Broadening the evaluation set. Results are restricted to DragBench — the paper does not report the size of the subset used — so generalization to other image distributions and drag patterns remains open.
• Explaining the redundancy result. The paper suggests neighboring timesteps are highly correlated, making multi-timestep supervision redundant; testing timesteps spaced further apart, rather than {30, 35, 40}, would probe whether that explanation is the full story.
• Tightening reproducibility. GPU non-determinism and evolved dependency versions still prevent exact numerical equality. Establishing what degree of agreement counts as a successful reproduction, and whether determinism can be recovered without a large performance penalty, is an open practical question.

Target Audience

This paper is most useful to researchers and engineers who intend to build on or deploy DragDiffusion and need to know which settings actually matter. It also serves reproducibility-methodology audiences and TMLR-style reviewers, since it models how to cross-reference reproduced experiments against the original figures and tables. Practitioners with intermediate diffusion-model familiarity will get the most out of the ablation tables; beginners can follow the claim-by-claim structure but may need background on DDIM inversion, latent optimization, and LoRA before the numbers become meaningful.

Authors’ abstract

DragDiffusion is a diffusion-based method for interactive point-based image editing that enables users to manipulate images by directly dragging selected points. The method claims that accurate spatial control can be achieved by optimizing a single diffusion latent at an intermediate timestep, together with identity-preserving fine-tuning and spatial regularization. This work presents a reproducibility study of DragDiffusion using the authors' released implementation and the DragBench benchmark. We reproduce the main ablation studies on diffusion timestep selection, LoRA-based fine-tuning, mask regularization strength, and UNet feature supervision, and observe close agreement with the qualitative and quantitative trends reported in the original work. At the same time, our experiments show that performance is sensitive to a small number of hyperparameter assumptions, particularly the optimized timestep and the feature level used for motion supervision, while other components admit broader operating ranges. We further evaluate a multi-timestep latent optimization variant and find that it does not improve spatial accuracy while substantially increasing computational cost. Overall, our findings support the central claims of DragDiffusion while clarifying the conditions under which they are reliably reproducible. Code is available at https://github.com/AliSubhan5341/DragDiffusion-TMLR-Reproducibility-Challenge.