PSR: Scaling Multi-Subject Personalized Image Generation with Pairwise Subject-Consistency Rewards

AI summary

Overview

Research area: Computer vision, specifically personalized (subject-driven) image generation, multi-subject composition, dataset construction, and reinforcement learning for diffusion/flow-matching models.

Technical level: Advanced. The paper assumes familiarity with diffusion/flow-matching generative models, LoRA fine-tuning, GRPO-style policy optimization, positional encodings in transformer image models, and standard evaluation tools such as DINO, CLIP, and MLLM-based judging.

Scope: The paper proposes a scalable data pipeline, a two-stage training recipe (supervised fine-tuning plus multi-reward reinforcement learning), and a new seven-subset benchmark for generating images containing two to four personalized subjects from reference images.

What This Paper Is About

Personalized image generation works well for a single subject, but degrades when several subjects must appear together: models lose subject identity, omit subjects, or misassign attributes (for example, swapping "the dog wears a chef's hat and the cat wears a scarf"). The authors argue this stems from two causes: no high-quality multi-subject dataset, and no fine-grained post-training objective that supervises individual subjects rather than whole images. Their goal is to build such a dataset, add subject-level reward signals during training, and provide a benchmark that measures the problem fairly.

Key Contributions

1. A scalable multi-subject data generation pipeline. Large language models, text-to-image models, an object detector (GroundingDINO), and single-subject personalization models are chained together to synthesize paired data with an arbitrary number of subjects. With predefined cleaning strategies, this produces approximately 350K high-quality samples.
2. A two-stage training paradigm with new reward and encoding designs. A scalable frame-wise positional encoding extends a single-subject model to multi-image, multi-subject inputs during supervised fine-tuning, and Pairwise Subject-Consistency Rewards (PSR) are combined with semantic and aesthetic rewards in a reinforcement learning stage.
3. PSRBench. A fine-grained benchmark of seven subsets (Attribute, Background, Action, Position, Complex, Three, Four), each with 50 evaluation samples, scoring subject consistency, aesthetic preference, and semantic alignment.
4. State-of-the-art results across benchmarks. The method reports the best overall scores on PSRBench's three dimensions and on the DreamBench multi-ip subset, plus a user study in which it is ranked first.

Main Findings

• Subject consistency on PSRBench: "Ours (PSR)" reaches an overall score of 0.673, versus 0.559 for the SFT-only version and 0.587 for the strongest baselines (OmniGen2 and XVerse, both 0.587). It leads on every individual subset.
• Harder subsets show the largest gains: On the Three and Four subsets the model reaches 0.615 and 0.571, compared with previous best results of 0.552 and 0.508.
• Aesthetic preference: Overall 1.124 for "Ours (PSR)", compared with 0.794 for "Ours-SFT" and 1.020 for OmniGen2. The paper attributes the SFT model's drop (below FLUX.1 Kontext's 0.870) to a training-resolution mismatch: FLUX.1 Kontext is trained at 1024 resolution while SFT is mainly done at 512, and reports that the RL stage restores perceptual quality.
• Semantic alignment: Overall 0.783 for "Ours (PSR)" versus 0.761 for Qwen-Image-Edit-2509, 0.758 for OmniGen2, and 0.712 for "Ours-SFT". The paper highlights strength on the Position and Four subsets.
• Positional encoding ablation: Training variants for one epoch on 50K randomly sampled training examples, the proposed frame-wise encoding ("w/ ours") gives semantic scores of 0.922 (2 subjects), 0.870 (3 subjects), 0.821 (4 subjects) and 0.508 on the Position subset, beating encoding variants that offset along height and width. The text reports the lead on the Position subset over the second-best method as 0.39; the table values are 0.508 (ours) and 0.469 (w/ h-w).
• Additional baselines: On PSRBench, the method also outperforms MS-Diffusion and MOSAIC across subject consistency, aesthetic score, and semantic alignment (for example, overall subject consistency 0.673 versus 0.501 for MOSAIC and 0.339 for MS-Diffusion).
• DreamBench: On 100 randomly sampled DreamBench multi-ip test cases under the official UNO protocol, the method reports the best DINO score at 0.529 (table second-best 0.506 for OmniGen2; the text describes the margin as 0.23), with CLIP-T 0.335 and CLIP-I 0.717. The authors themselves note CLIP semantic evaluation is not always accurate and global DINO is susceptible to background interference.
• User study: With 100 test cases and five participants ranking four methods (scores 1, 0.8, 0.6, 0.4 from highest to lowest), the method scores 0.92 (subject consistency), 0.80 (semantic alignment), and 0.82 (human preference).
• Reward design matters: The paper reports that combining multiple rewards mitigates reward hacking, whereas PSR alone tends to induce copy-and-paste behavior.
• Reported failure mode: Identity preservation struggles for small subjects.

Methodology in Plain English

Building the data. For data with n subjects, the authors sample n categories from the Objects365 category pool, then ask a large language model (Qwen3-32B) to write both a text-to-image prompt and per-category subject-driven prompts. A text-to-image model (FLUX.1-schnell, with Qwen-Image used for four-subject cases) renders a scene containing all n subjects. GroundingDINO detects and crops each instance, and those crops plus subject instructions are fed to a single-subject personalization model (FLUX.1 Kontext) to produce new reference images. This yields paired, high-quality multi-subject data at roughly 350K samples.

Building the instructions. Because the original text-to-image prompts describe appearance, reusing them would let the model cheat by reading appearance from the text instead of the reference images. So instructions are regenerated with an MLLM across seven predefined task types (attribute, background, action, positional, complex-scene, three-subject, four-subject), using pronouns and task-specific recaptioning; attribute and action tasks get an extra re-editing pass. Cleaning then filters samples by paired DINO consistency and Qwen2.5-VL semantic alignment.

Training stage one (SFT). The base model is FLUX.1 Kontext tuned with LoRA (rank 512, learning rate 1e-4). Instead of UNO-style offsets in the height and width dimensions, the authors use a virtual temporal offset per input image, P O_i = (i, h, w), so the model is not forced to assume the second image sits to the right of or below the first. Training mixes 2, 3, and 4 reference images (sampling probabilities 0.9, 0.05, and 0.05) for joint generalization.

Training stage two (RL). Using GRPO as adapted for flow matching by Flow-GRPO (SDE sampling, group size 6, LoRA rank 64, learning rate 1e-5, 28 diffusion timesteps), the model is optimized with a weighted reward: PSR for subject consistency (0.4), a semantic reward from Qwen2.5-VL-32B-Instruct (0.4), and an aesthetic reward from HPSv3 (0.2). The core PSR idea is subject decoupling: an open-vocabulary detector crops each subject from both the generated image and the corresponding reference image (references may contain backgrounds), and the reward is the average DINO-feature similarity across corresponding subject pairs. Images are resized to 512×512 (or shortest side 512 for non-square inputs).

Why This Matters

Impact on research: The paper reframes multi-subject personalization as a data problem plus a reward-design problem, not just an architecture problem. Its subject-decoupling reward offers a general pattern for turning coarse image-level objectives into fine-grained, entity-level supervision, and PSRBench's seven subsets and explicit evaluation protocol give the field a stricter alternative to global DINO or CLIP scores.

Real-world applications (the paper explicitly names film production and personalized marketing; the others are natural extensions of multi-subject reference-image editing):

• Film and video pre-production, storyboarding scenes where recurring characters must stay recognizable.
• Personalized marketing and advertising, placing a specific product alongside a specific person or mascot in new scenes.
• E-commerce catalog imagery, composing several named products into one lifestyle scene while preserving each item's appearance.
• Children's books, comics, and other narrative illustration, where several characters must appear together in prompted poses and settings.

Industry relevance: The pipeline reuses existing single-subject models rather than training new ones, which is an economical route to scaling data. The reported failure mode (small subjects lose identity) and the reward-hacking observation are practical warnings for anyone building RL post-training loops around generative models.

Future Directions

• Improve small-subject identity preservation, the failure case the authors identify, and quantify how consistency degrades as subject count grows beyond four.
• Reduce reliance on imperfect automatic metrics. The authors criticize CLIP-based semantic scoring and background-sensitive global DINO, and their own Position metric required an added grounding-based check, so better fine-grained evaluation remains open.
• Resolve the resolution question. The SFT model's aesthetic score fell relative to FLUX.1 Kontext at 512 resolution versus the base model's 1024, which suggests resolution-matched training as an obvious next experiment.
• Extend the reward and data pipeline to more subjects, richer inter-subject interactions (the paper shows some interaction cases), and possibly to video or other modalities, with stricter safeguards against reward hacking.

Target Audience

Researchers and engineers working on subject-driven image generation, diffusion/flow-matching post-training, and reinforcement learning for generative models. It is also useful for practitioners building product or character-consistency pipelines who need a concrete data-construction recipe, and for benchmark designers interested in how to score multi-subject outputs per subject rather than per image. Readers without a background in diffusion sampling and policy-gradient methods will find the methodology sections demanding.

Authors’ abstract

Personalized generation models for a single subject have demonstrated remarkable effectiveness, highlighting their significant potential. However, when extended to multiple subjects, existing models often exhibit degraded performance, particularly in maintaining subject consistency and adhering to textual prompts. We attribute these limitations to the absence of high-quality multi-subject datasets and refined post-training strategies. To address these challenges, we propose a scalable multi-subject data generation pipeline that leverages powerful single-subject generation models to construct diverse and high-quality multi-subject training data. Through this dataset, we first enable single-subject personalization models to acquire knowledge of synthesizing multi-image and multi-subject scenarios. Furthermore, to enhance both subject consistency and text controllability, we design a set of Pairwise Subject-Consistency Rewards and general-purpose rewards, which are incorporated into a refined reinforcement learning stage. To comprehensively evaluate multi-subject personalization, we introduce a new benchmark that assesses model performance using seven subsets across three dimensions. Extensive experiments demonstrate the effectiveness of our approach in advancing multi-subject personalized image generation. Github Link: https://github.com/wang-shulei/PSR