Personalized Image Editing in Text-to-Image Diffusion Models via Collaborative Direct Preference Optimization

AI summary

Overview

Research area: Computer Vision — personalized text-to-image (T2I) diffusion model editing and preference alignment.

Technical level: Intermediate. The paper assumes familiarity with diffusion models, LoRA fine-tuning, and Direct Preference Optimization (DPO), but its central ideas (learning per-user aesthetic preferences and sharing them across similar users) are explained in accessible terms.

Scope: The paper introduces Collaborative Direct Preference Optimization (C-DPO), a training framework that makes text-to-image editing models adapt to individual users' aesthetic preferences while borrowing preference signals from like-minded users through a graph neural network.

What This Paper Is About

Text-to-image diffusion models are trained to satisfy an average user, so they do not adapt to the specific aesthetic tastes of an individual — one user may want brighter, more saturated colors while another prefers muted tones and centered composition. This paper defines the new task of personalized image editing, in which an editing model is tailored to a single user's preferences, and proposes a method that also lets users with overlapping tastes learn from each other's edit histories.

Key Contributions

1. The first formulation of personalized text-to-image editing. The authors define a new problem setting where an editing model is tailored to an individual user's preferences, moving beyond the one-size-fits-all paradigm in image editing.

2. Collaborative Direct Preference Optimization (C-DPO). A training framework that adds a graph-structured regularization term to the DPO loss, explicitly modeling collaborative relationships among learned user embeddings so that preferences implicitly shared among like-minded users can be captured.

3. A synthetic benchmark dataset of 144K editing preferences. The dataset contains 3,000 synthetic user profiles with annotated edits grouped by user, spanning individual like/dislike preferences, providing a benchmark for studying personalization in image editing.

4. Public release of source code and dataset at http://personalized-editing.github.io.

Main Findings

• Prompt-level alignment favors C-DPO. Using CLIP-text similarity between generated instructions and users' ground-truth liked instructions, the method scored highest across all three profile-completeness conditions: 0.354 (± 0.068) with Like+Dislike, 0.306 (± 0.069) with Likes only, and 0.294 (± 0.071) with Dislikes only. The next best model, DPO-User, scored 0.307 (± 0.071), 0.286 (± 0.071), and 0.269 (± 0.076) respectively.

• Generic baselines lag well behind. Qwen-VL scored 0.274 (± 0.055), 0.276 (± 0.052), and 0.229 (± 0.051); LLaVA scored 0.244 (± 0.065), 0.254 (± 0.061), and 0.227 (± 0.051); SFT-Vanilla scored 0.276 (± 0.049), 0.276 (± 0.042), and 0.185 (± 0.065); SFT-Pix2Pix scored 0.227 (± 0.078), 0.238 (± 0.065), and 0.196 (± 0.065); and DPO-Vanilla scored 0.272 (± 0.059), 0.274 (± 0.057), and 0.172 (± 0.057).

• Better preservation of the original image. Comparing original inputs with personalized edits, C-DPO achieved DINO-Ref 0.782 (± 0.182), CLIP-Ref 0.652 (± 0.182), HPS 0.249 (± 0.034), and CLIP-T 0.358 (± 0.043). DPO-User reached 0.762, 0.649, 0.243, and 0.354; DPO-Vanilla 0.736, 0.615, 0.242, and 0.346; and SFT 0.690, 0.600, 0.238, and 0.331.

• Strongest alignment with liked images and liked prompts. Against the user's liked images and prompts, C-DPO scored DINO-Pref 0.059 (± 0.106), CLIP-I-Pref 0.338 (± 0.089), CLIP-T-Pref 0.364 (± 0.038), and a neighbor-alignment score CLIP-T-Neighbor of 0.312 (± 0.020). The CLIP-T-Neighbor measure compares generated images against attributes aggregated from each user's ten most similar users in the preference graph.

• Human judges prefer the method by a large margin. In a crowd-sourced study on Prolific.com with 50 participants judging 10 synthetic user profiles from the test split, C-DPO won 0.616 of Q1 (general preference, "Which edited image would this user prefer, given the prompt?") and 0.465 of Q2 (visual appeal). DPO-Vanilla received 0.210 and 0.210; DPO-User received 0.174 and 0.325.

• Outperforms a personalized generation baseline. Against VIPER, C-DPO scored CLIP-I 0.326 (± 0.020), DINO 0.037 (± 0.016), and HPS 0.253 (± 0.046), versus 0.286 (± 0.015), 0.035 (± 0.008), and 0.205 (± 0.050) for ViPer.

• Outperforms recent instruction-based editing frameworks. With identical user-preference text prompts, C-DPO reached CLIP-I-Pref 0.3515 (± 0.0190), DINO-Pref 0.0365 (± 0.0164), and HPS 0.2607 (± 0.0328), compared with BAGEL (0.3397 ± 0.0144, 0.0363 ± 0.0066, 0.2475 ± 0.0388), SEED-Llama (0.3109 ± 0.0159, 0.0349 ± 0.0121, 0.2500 ± 0.0287), and Nexus-Gen (0.3327 ± 0.0138, 0.0373 ± 0.0113, 0.2598 ± 0.0307).

• Efficient to train and run. The SFT stage trains one epoch on the InstructPix2Pix training set on 2 GPUs in just under 3 hours; C-DPO trains 3,000 steps on 3 GPUs in approximately 1 hour. Each personalized edit takes around one minute, or about one second with a faster backend such as TurboEdit.

Methodology in Plain English

The system is built in three stages.

Teaching a language model to write precise edit instructions. The authors fine-tune QWEN2.5 with a LoRA adapter on paired base-image and editing-instruction data from InstructPix2Pix, teaching it to turn a high-level cue into a concrete instruction. The resulting adapter is frozen and becomes the reference policy.

Modeling users as a graph. Each user becomes a node in a heterogeneous, bipartite, undirected user–preference graph. Attribute nodes represent editing topics such as color palettes and textures and are initialized with text embeddings from a pretrained language model; edges connect users to attributes they like or dislike. A lightweight graph neural network — a 2-layer GraphSAGE encoder with mean aggregation and a two-layer linear decoder — aggregates neighborhood information into a contextualized embedding per user. It is pretrained on an edge-classification task that predicts whether a user–attribute link is positive or negative, using 60% of the edges for training and 20% each for validation and testing. Because GraphSAGE aggregates over a node's neighborhood, new users can be added at inference with a zero-initialized feature vector and connected to liked and disliked attributes, without retraining.

A collaborative preference loss. User similarity is measured by the normalized number of shared one-hop neighbors. The C-DPO loss adds two terms: the standard DPO loss for the user themselves, plus a weighted average of DPO losses computed for that user's K nearest graph neighbors, scaled by a collaboration strength λ (set to 0.15). Because the reference policy stays user-agnostic, the collaborative term pulls the policy toward neighbor preferences only where the data supports it.

Injecting the user into the model. Each user's graph embedding passes through a two-layer MLP and becomes a fixed set of soft prompt tokens prepended to the editing instruction, so no change to the base language model architecture is needed. The GNN and MLP train jointly with a second LoRA adapter (the policy), with a lower learning rate on the GNN to preserve its pretrained structure.

Producing the actual image. The personalized prompt is handed to FLUX.1-dev augmented with ControlNet (InstantX FLUX.1-dev-Controlnet-Union), using a conditioning scale of 0.4 and Canny as the condition.

Data. With no existing dataset for this task, the authors generated 3,000 synthetic user profiles spanning 144K samples, using demographic configurations along axes such as age, geography, and socioeconomic status. Each user is assigned four randomly sampled image captions drawn from 80 COCO categories; for each base image, two pairs of preferred and rejected editing instructions are generated across six edit types, yielding 48 annotated editing instructions per user. Roughly 2,900 users form the training split and 100 are held out for testing.

Why This Matters

Impact on research. The paper reframes image editing from a single global objective into a multi-user preference-learning problem, and shows that collaborative signals can be added to DPO through graph regularization without sacrificing per-user customization. It also supplies a benchmark dataset for a task that previously had none, which lets other researchers measure personalization quantitatively rather than by anecdote.

Real-world applications.

• Consumer photo editing and creative tools that learn a person's recurring visual style instead of requiring repeated prompt engineering.
• Interior design and home-decor workflows, where a user who routinely adds a stone fireplace and distressed-leather sofa to make a rustic living room can have related details such as exposed wooden ceiling beams proposed automatically from similar users' histories.
• Personalized generation beyond editing, which the authors demonstrate by benchmarking against VIPER.
• Accessibility support: the authors note the framework can help non-experts, including artists with motor impairments or limited technical skills, reach their desired edits more efficiently.

Industry relevance. The framework is editing-model agnostic: it pairs with FLUX.1-dev plus ControlNet, but can also use a faster backend such as TurboEdit, cutting edit time from about one minute to roughly one second. It trains on a single NVIDIA L40 (48GB) GPU, and the ability to add unseen users without retraining is directly relevant to deployed products that must scale to new accounts continuously.

Stated risks. Tailoring outputs to inferred tastes risks reinforcing aesthetic "filter bubbles" and narrowing exposure to diverse visual styles. Users with neither personal edit history nor close graph neighbors receive only generic editing, and any bias in the Flux and ControlNet backbones can propagate into the edits.

Future Directions

• Video personalization. The authors identify extending the framework beyond still images to the video domain, where temporal coherence and multi-object consistency introduce new challenges, as an exciting follow-up direction.

• Real-world preference data. The dataset is synthetic because of scalability and budget constraints on crowd-sourcing; gathering more authentic and nuanced real user preference data is named as an exciting future research direction.

• Periodic graph refinement. The paper notes that periodic fine-tuning as the graph expands may further improve embedding quality, though it is not required for inference — leaving open how best to schedule such updates.

• Handling cold-start users. The limitation that a new user with no personal edits and no close neighbors defaults to generic editing raises the question of how to personalize in the absence of both signals.

Target Audience

Researchers and practitioners working on text-to-image diffusion models, preference alignment, or recommendation-style personalization will get the most from this paper, as will engineers building consumer-facing generative editing products that need to adapt per user. Readers interested in graph neural networks applied to multimodal preference modeling, and those who need a benchmark for evaluating personalization in image editing, will also find it useful.

Authors’ abstract

Text-to-image (T2I) diffusion models have made remarkable strides in generating and editing high-fidelity images from text. Yet, these models remain fundamentally generic, failing to adapt to the nuanced aesthetic preferences of individual users. In this work, we present the first framework for personalized image editing in diffusion models, introducing Collaborative Direct Preference Optimization (C-DPO), a novel method that aligns image edits with user-specific preferences while leveraging collaborative signals from like-minded individuals. Our approach encodes each user as a node in a dynamic preference graph and learns embeddings via a lightweight graph neural network, enabling information sharing across users with overlapping visual tastes. We enhance a diffusion model's editing capabilities by integrating these personalized embeddings into a novel DPO objective, which jointly optimizes for individual alignment and neighborhood coherence. Comprehensive experiments, including user studies and quantitative benchmarks, demonstrate that our method consistently outperforms baselines in generating edits that are aligned with user preferences.