Qwen-Image-PE: Learning Visual Design for Text-to-Image Prompt Enhancement

AI summary

Overview

Research area: Computer Vision / generative AI — specifically prompt enhancement (PE) for text-to-image (T2I) generation, with an emphasis on learning fine-grained visual design (lighting, composition, layout, detail).

Technical level: Intermediate. Readers should be comfortable with supervised fine-tuning, reinforcement learning for LLMs (GRPO), and how a T2I generator is conditioned on text; the paper itself is written in a fairly readable style.

Scope: The paper proposes Qwen-Image-PE, a two-stage trained prompt enhancer (9B and 397B variants) that turns brief user requests into detailed prompts encoding explicit visual design choices, and it reports benchmark, ablation, scaling, and cross-backbone transfer results on Qwen-Image-Bench and four additional T2I backbones.

What This Paper Is About

T2I models can follow very detailed prompts, but users typically write short requests that leave design decisions — where the light comes from, how the frame is composed, how text and elements are laid out, how rich the textures are — completely unspecified. Prompt enhancement bridges that gap by expanding a short request into a long prompt, and the design choices made during that expansion determine how the requested content actually looks.

The paper's goal is to make prompt enhancement explicitly learn fine-grained visual design rather than only preserving intent or optimizing generic "visual appeal," and to show that this capability scales and transfers across different image generators.

Key Contributions

1. A two-stage training paradigm for prompt enhancement. Stage one is supervised fine-tuning with Caption-Based Reverse Distillation (CRD), which derives enhanced-prompt targets by densely captioning well-designed images (reversing the usual text-to-image direction) and trains the enhancer to predict those captions from user requests. Stage two is reinforcement learning with Groupwise Design Reward (GDR), which scores all candidate images sampled for the same request jointly in a grid under explicit design rubrics.

2. A groupwise reward design that is cheaper than alternatives. GDR evaluates the whole rollout group of G candidate images in a single call per design dimension, using K shuffled grids, requiring D·K judge calls per group versus D·G for pointwise scoring and D·(G choose 2) for exhaustive pairwise scoring, with K much smaller than G.

3. Scaling analysis of prompt enhancement along three axes. The paper reports that PE performance scales with training data, model size, and information density, and argues that sufficient information density is a prerequisite for effective enhancement — low-density supervision actually degrades below the original request.

4. Generalization evidence across T2I backbones. Both PE variants improve on original requests on four additional frozen backbones without access to their training captions, and surpass the native prompt enhancers on HunyuanImage-2.1 and FLUX 3 Image Klein. The 9B model is released at https://huggingface.co/Qwen/Qwen-Image-2.1-PE-T2I.

Main Findings

• Best Overall on Qwen-Image-Bench with a shared Qwen-Image-2.1 backbone. Qwen-Image-PE-397B reaches an Overall score of 62.17 versus 54.36 for Original Request, a gain of 7.81 points, and leads all five dimensions (Quality, Aesthetics, Alignment, Real-world Fidelity, Creative Generation).

• The 9B model beats every evaluated PE baseline. Qwen-Image-PE-9B scores 60.28 Overall, compared with 60.10 for GPT-5.5, 59.90 for FLUX 3 Image Klein PE, 59.59 for Qwen-3.8-Max, 60.04 for Gemini-3.1-Pro, 58.70 for ERNIE-Image-PE, 57.13 for HunyuanImage-2.1-PE-32B, and 55.96 for HunyuanImage-2.1-PE-7B.

• Fine-grained design gains, with one near-tie. Qwen-Image-PE-397B leads on Lighting (76.59), Composition (66.66), and Detail (58.91), while its Layout score of 61.11 is just below GPT-5.5's 61.15. Qwen-Image-PE-9B (75.51 / 63.80 / 60.83 / 58.18) outperforms all trained baselines on the four design dimensions.

• Aesthetics and image quality improve too. Both variants beat all baselines on the three UniPercept metrics (IAA, IQA, ISTA), HPSv3, and PickScore; the 397B model ranks first on eight of nine metrics in that table.

• Human preference favors the 397B model. In GSB evaluation on 500 expert-annotated cases, net win rates for Qwen-Image-PE-397B are 35.48% over Original Request, 32.30% over HunyuanImage-2.1-PE-32B, 22.49% over ERNIE-Image-PE, 17.09% over GPT-5.5, and 13.60% over FLUX 3 Image Klein PE.

• CRD beats forward-based supervision. With identical settings and the same MLLM used to build targets, CRD raises Overall from 59.86 to 61.52 (+1.66) over forward-based supervision, with the largest gains in Aesthetics (+3.00) and Creative Generation (+2.42).

• CRD produces prompts closer to the caption distribution. Compared with Qwen-3.8-Max, Qwen-Image-PE-397B reduces unigram/bigram JS divergence from 0.229/0.539 to 0.105/0.329, reduces Fréchet distance from 0.0709 to 0.0207, and raises MAUVE from 0.032 to 0.414.

• GDR beats pointwise design reward. Starting from the CRD-trained 397B model (61.52), pointwise design reward drops to 61.36 Overall, while GDR reaches 62.17 — 0.81 above pointwise and 0.65 above the initialization, with the largest dimension gain in Aesthetics (+1.70).

• GDR's ranking agrees with humans more often. Reward orderings match blinded human judgments on rollout image pairs 59.1% of the time for GDR versus 50.9% for pointwise scoring, an 8.2 percentage-point difference.

• Training-time behavior on the benchmark. During RL training of Qwen-Image-PE-397B, Lighting shows the most consistent improvement, rising from 72.14 to 77.64; Composition and Detail trend positive but noisier, and Layout improves only slightly with substantial fluctuations.

• Data scaling with diminishing returns. On nested subsets of 12.5K, 25K, 50K, and 100K examples with a Qwen3.5-9B enhancer, Overall rises 55.87 → 56.68 → 57.31 → 57.61, a total gain of 1.74 points, with per-doubling gains shrinking from 0.81 to 0.63 to 0.30.

• Model scaling is the largest sweep effect. With training data fixed at 50K, Overall rises monotonically from 49.56 (Qwen3.5-2B) to 59.86 (Qwen3.5-397B-A17B) across five dense and MoE models, a 10.30-point gain, with the largest dimension gains in Creative Generation (+19.66) and Alignment (+14.80).

• Information density dominates, and low density can hurt. With the model fixed at Qwen3.5-9B and data at 50K, Overall rises from 44.65 to 51.36, 55.39, and 57.31 across four density levels (L1–L4), a 12.66-point gain. L1 and L2 fall below the Original Request baseline of 54.36, while L3 and L4 exceed it by 1.03 and 2.95 points.

• Transfer across four backbones without their captions. On FLUX.2-dev, Krea 2 Turbo, HunyuanImage-2.1, and FLUX 3 Image Klein, both variants improve on original requests with gains across all five dimensions: 2.53–6.74 points for the 9B model and 3.86–8.03 points for the 397B model. Qwen-Image-PE-397B exceeds HunyuanImage-2.1-PE-32B by 3.07 points and FLUX 3 Image Klein PE by 2.74 points.

Methodology in Plain English

The enhancer takes a user request and outputs three things: a chain-of-thought "design rationale," a detailed enhanced prompt, and an aspect ratio. That prompt then conditions a frozen T2I generator that produces the actual image. Training happens in two stages.

Stage one — learning by imitating well-designed images. Rather than asking a language model to freely expand requests, the authors work backwards from images. They assemble request–image pairs two ways: generating images from real and synthetic requests and keeping the ones that look well designed and consistent with the request, or starting from quality-filtered images and degrading their short captions into noisier, less structured requests. A multimodal captioner then writes a dense description of each selected image covering lighting, composition, layout, and detail; that caption becomes the target "enhanced prompt," and the image's aspect ratio becomes the target aspect ratio. A language model then writes a rationale connecting the original request to that caption. The enhancer is fine-tuned to produce the rationale, prompt, and aspect ratio from the request alone.

Stage two — learning by comparison. For each request in an RL prompt pool (mined where the SFT model produced images with design weaknesses, plus synthesized requests), the enhancer samples a group of G responses, and the frozen generator renders an image for each using a shared sampling seed. All G images are placed in a grid and a multimodal judge scores them together against a rubric for each design dimension, conditioning each candidate's score on the whole group. The grids are shuffled K times to reduce order sensitivity, scores are normalized to [0, 1] and averaged across passes. Those design rewards are combined with an image–request alignment reward to preserve the user's explicit requirements, and the policy is optimized with GRPO with KL regularization toward a frozen copy of the SFT policy.

Why this design. The authors motivate CRD with two observations: dense captions of well-designed images do guide Qwen-Image-2.1 to reproduce key design choices, and in a t-SNE projection using Qwen-Image-2.1 text-encoder embeddings, CRD-derived prompt embeddings overlap more with 1,000 sampled T2I training captions than forward-enhanced prompts (from Qwen-Image-PE-397B and Qwen-3.8-Max on the same 1,000 Qwen-Image-Bench requests) do. They motivate GDR by arguing that pointwise judging lacks shared visual references and exhaustive pairwise judging is too expensive for online RL.

Evaluation. Comparisons use Qwen-Image-2.1 as a shared frozen backbone at 2K resolution with matched per-request sampling seeds. Evaluation requests do not overlap with SFT or RL data, and the evaluation metrics are distinct from training rewards. Besides Qwen-Image-Bench, they use UniPercept (IAA, IQA, ISTA), HPSv3 and PickScore on GenEval prompts (chosen because those metrics have text-length limits), GSB human evaluation, and a human-agreement study on reward rankings.

Why This Matters

For research: The paper reframes prompt enhancement from "preserve intent and raise generic appeal" to "explicitly learn transferable design competence," and it supplies evidence that the supervision signal's information density — not just its quantity — determines whether enhancement helps or hurts. The finding that low-density supervision performs worse than no enhancement at all (44.65 and 51.36 versus 54.36) is a caution for anyone building prompt-rewriting pipelines. The cross-backbone transfer result, including beating native enhancers on two generators without their training captions, argues against the view that prompt enhancers merely learn generator-specific caption quirks.

Real-world applications:

• Consumer and prosumer image tools: automatically turning a one-line request into a prompt that produces professional-looking lighting and composition without the user learning prompt engineering.

• Marketing and e-commerce asset generation: product shots, banners, and social creatives where layout, whitespace, and typographic hierarchy are explicit requirements rather than afterthoughts.

• Design and presentation mockups: requests that involve organized content modules and readable hierarchy benefit directly from the Layout dimension the paper optimizes.

• Multi-model pipelines: a single released enhancer (Qwen-Image-PE-9B) that improves output on backbones other than the one it was trained against, useful when teams switch or mix generators.

Industry relevance: Prompt enhancement is a cheap lever — it improves outputs from a fixed, frozen generator without retraining the image model, which is exactly the cost profile that serving teams want. The paper also quantifies the tradeoff: model size from 2B to 397B-A17B buys 10.30 Overall points, while going from 12.5K to 100K training examples buys only 1.74, which is directly actionable for budgeting training runs.

Future Directions

• Push on the dimension that resisted improvement. Layout improves only slightly with substantial fluctuations during RL training and is the one fine-grained dimension where the 397B model does not lead (61.11 vs. GPT-5.5's 61.15). What supervision or reward design would make layout as responsive as lighting, which rose from 72.14 to 77.64?

• Understand and raise the information-density threshold. The paper shows L1 and L2 supervision underperform the original request but does not characterize what makes a caption "dense enough." Identifying the threshold, or a way to train robustly below it, is an open question.

• Reduce reliance on a single frozen backbone during training. Training used Qwen-Image-2.1, and transfer to FLUX.2-dev, Krea 2 Turbo, HunyuanImage-2.1, and FLUX 3 Image Klein was tested only at evaluation time. Whether training against a mixture of generators improves transfer further is not reported.

• Replace or scale the MLLM judge. GDR uses D·K judge calls per group and deliberately avoids exhaustive pairwise judging as impractical at the authors' training scale. Cheaper or more reliable judging, and validating the 59.1% human-agreement level further, are natural next steps. The paper also does not report a formal limitations section, so the boundaries of the approach under out-of-distribution request types remain unstated.

Target Audience

Researchers and engineers working on T2I systems, prompt optimization, and multimodal RL — particularly those who build prompt-rewriting layers on top of frozen image generators and want to know which levers (data, model size, caption density) actually move quality. It is also useful for practitioners in design tooling and creative-asset generation who care about lighting, composition, layout, and detail as separable targets, and for RL researchers interested in groupwise, MLLM-judged reward design as an alternative to pointwise and pairwise scoring. Readers without background in RL fine-tuning will still follow the motivation and results, but the training sections assume familiarity with policy optimization and reward shaping.

Authors’ abstract

Learning visual design is essential for prompt enhancement (PE) to turn underspecified user requests into coherent visual specifications for text-to-image (T2I) generation. Existing methods preserve user intent and improve overall visual appeal, with less explicit emphasis on fine-grained design dimensions. To address this gap, we introduce Qwen-Image-PE, which learns visual design through two-stage training. Supervised fine-tuning with Caption-Based Reverse Distillation (CRD) uses dense captions of well-designed images as design supervision. Reinforcement learning with Groupwise Design Reward (GDR) jointly compares all candidate images for the same request to refine lighting, composition, layout, and detail. On Qwen-Image-Bench with Qwen-Image-2.1, Qwen-Image-PE-397B improves over original requests by 7.81 points, while Qwen-Image-PE-9B outperforms all evaluated PE baselines. Further evaluations show gains in visual design, aesthetics, image quality, and human preference. We find that PE scales along three axes: training data, model size, and information density, with sufficient information density as a prerequisite for effective enhancement. Moreover, we demonstrate that Qwen-Image-PE generalizes to four additional T2I backbones, even surpassing native prompt enhancers on HunyuanImage-2.1 and FLUX 3 Image Klein. We release Qwen-Image-PE-9B at https://huggingface.co/Qwen/Qwen-Image-2.1-PE-T2I.