AutoPP: Towards Automated Product Poster Generation and Optimization

AI summary

Overview

Research area: Computer vision and generative AI applied to e-commerce advertising — specifically automated product poster creation and click-through-rate (CTR) optimization.

Technical level: Advanced. The paper assumes familiarity with diffusion/flow-matching generative models, multimodal large language models (MLLMs), parameter-efficient fine-tuning (LoRA), decomposable attention in MM-DiT blocks, and Direct Preference Optimization (DPO). The core ideas are accessible, but the implementation details are dense.

Scope: The paper proposes AutoPP, an end-to-end pipeline that generates product posters from minimal input and then optimizes them for real-world CTR using pairwise feedback, supported by a new one-million-poster dataset.

What This Paper Is About

Manually designing product posters and then iteratively tweaking them based on online performance is slow and expensive. Prior automated methods still require humans to supply layouts, selling points, or text-rendering rules, and they only optimize a single poster element (usually the background) for engagement. AutoPP aims to remove both human bottlenecks by (1) generating a complete poster — background, text, and layout — from just a product image and candidate text, and (2) automatically improving CTR by learning which isolated element actually drives user clicks.

Key Contributions

1. An automated generator that takes only basic product information (a product image plus candidate text) and jointly produces all three poster elements — background prompt, selected text, and layout — through a single unified design module, avoiding the fragmented multi-model pipelines of prior work.
2. An element rendering module that encodes glyphs and product images as condition tokens and uses a decomposed attention mechanism (condition self-attention plus image-condition cross-attention) to generate the final poster efficiently and controllably, without requiring pixel-level alignment between glyph and target images.
3. An automated optimizer combining a systematic element-replacement strategy with Isolated Direct Preference Optimization (IDPO), which attributes CTR gains to isolated poster elements rather than treating the poster as a single holistic output.
4. AutoPP1M, described as the largest product poster dataset to date: one million high-quality posters with element-level annotations plus CTR feedback from over 1.1 million users and 50,000 pairwise comparisons.

Main Findings

• State-of-the-art generation quality: AutoPP achieves the best scores on overall poster quality and layout rationality (FID 60.71, Clip-T 29.75, Alignment 0.007, Overlap 0.011, MIoU 0.256), outperforming P&R, PAID, and GPT-4o. GPT-4o in particular struggles with Chinese text and produces stroke errors.

• Superior text rendering: In a controlled comparison using identical layouts and prompts, AutoPP reaches 65.19 sentence accuracy and 12.94 normalized edit distance, beating FLUX-Fill, FLUX-ControlNet, Glyph-byt5-v2, AnyText-v2, and PosterMaker — especially in small-font scenarios.

• Positive online CTR gains: In a one-week live experiment on JD.COM with 10,000 products, AutoPP produced a +4.49% relative CTR improvement over the unoptimized generator, while CG4CTR and CAIG actually showed negative relative CTR because they ignore text and layout.

• Element-isolated optimization beats holistic DPO: Using identical training data, IDPO (4.49% CTR gain) outperforms standard DPO (3.10%), showing that attributing feedback to specific elements yields more efficient learning.

• Data scaling helps: Reward accuracy rises steadily with dataset size — 51.20% at 10K samples, 67.19% at 30K, and 75.99% at 50K — with no sign of saturation.

• Efficiency without extra parameters: Adding product and text control adds zero new parameters (PosterMaker adds 1.6B, FLUX-ControlNet adds 4.2B). The decomposed attention mechanism cuts GFLOPs by 18% at 800×800 and 24% at 1024×1024 with comparable quality (65.19 vs 66.37 Sen. Acc; 43.19 vs 44.28 FID).

• Robustness and flexibility: Token-based glyph encoding tolerates misalignment between glyph and target images, and the model generalizes emergently to English, Japanese, and Korean glyphs despite being trained mainly on Chinese. It also handles multiple aspect ratios (e.g., 4:3, 16:9) and can produce several distinct layouts for the same product.

Methodology in Plain English

The pipeline has two halves. The generator starts with a unified design module — a multimodal LLM fine-tuned from LLaVA — that receives the product image plus candidate text and autoregressively outputs three things at once: a background prompt, the text to use, and a layout specifying where elements go. The element rendering module, built on FLUX.1 dev, then turns those decisions into pixels. It encodes the product image, a glyph image (text drawn at the predicted positions), and the target image into token sequences, then runs them through modified MM-DiT blocks. Rather than concatenating everything into one expensive full-attention pass, it uses decomposed attention: self-attention within glyph and product tokens, and cross-attention where prompt and noise queries attend to all condition tokens. Training combines a standard flow-matching loss with an OCR perceptual loss that directly penalizes illegible text.

The optimizer works by systematically creating poster variants — swapping only the background prompt, only the text (with length-matched alternatives), or only the layout — while holding everything else constant. These paired variants are shown to real users in a randomized experiment on JD.COM, producing fine-grained CTR feedback. IDPO then modifies the DPO objective by weighting each token according to which element it belongs to, so the model learns that, for example, a background change caused the click difference. Fine-tuning is applied only to the design module, using an element-specific weight (α = 5 for the replaced element, 1 for the rest).

Why This Matters

Research impact: This is the first framework to unify product poster generation and performance optimization in a single pipeline, and the first to introduce element-isolated preference learning for visual advertising. The AutoPP1M dataset — an order of magnitude larger than prior poster datasets and uniquely annotated at the element level — opens a new research direction in fine-grained preference modeling for images that contain text.

Real-world applications:

• E-commerce ad automation: Platforms can auto-generate and self-improve product posters at scale, eliminating design bottlenecks for millions of listings.
• Small business marketing: Merchants without design teams gain access to professional-quality, self-optimizing promotional visuals.
• Multilingual and responsive campaigns: The model's cross-lingual generalization and multi-aspect-ratio support suit international marketplaces and multi-channel ad formats.
• Personalized product presentation: The multi-layout capability can be extended to serve different user segments with tailored poster variants.

Industry relevance: The work comes from JD.COM and is validated in a live production environment. The authors note that even a 0.5% CTR improvement can yield significant financial returns at e-commerce scale, making the 4.49% gain commercially meaningful. The zero-extra-parameter design also matters for deployment cost.

Future Directions

• Personalized preference learning: Current optimization uses aggregated CTR across all users, which may under-serve minority groups; the authors propose moving toward individualized feedback mechanisms.
• Unified autoregressive architecture: Merging the design and rendering modules into a single autoregressive model would enable end-to-end optimization with RLHF rather than fine-tuning one module at a time.
• Richer preference signals: Beyond click-through rate, incorporating dwell time, conversion, or aesthetic judgments could broaden the optimization objective.
• Scaling and generalization: Given that reward accuracy continues to improve with data up to 50K samples, exploring much larger feedback sets and additional languages or product categories is a natural extension.

Target Audience

This paper is most valuable to computer vision and generative AI researchers working on controllable image synthesis, text rendering, or preference-based optimization. It is also directly relevant to applied scientists and engineers in e-commerce, advertising technology, and marketing automation who need production-ready poster generation. Practitioners interested in DPO variants and multimodal LLM fine-tuning will find the IDPO formulation and the decomposed-attention design particularly useful, while dataset-focused researchers will benefit from the AutoPP1M release.

Authors’ abstract

Product posters blend striking visuals with informative text to highlight the product and capture customer attention. However, crafting appealing posters and manually optimizing them based on online performance is laborious and resource-consuming. To address this, we introduce AutoPP, an automated pipeline for product poster generation and optimization that eliminates the need for human intervention. Specifically, the generator, relying solely on basic product information, first uses a unified design module to integrate the three key elements of a poster (background, text, and layout) into a cohesive output. Then, an element rendering module encodes these elements into condition tokens, efficiently and controllably generating the product poster. Based on the generated poster, the optimizer enhances its Click-Through Rate (CTR) by leveraging online feedback. It systematically replaces elements to gather fine-grained CTR comparisons and utilizes Isolated Direct Preference Optimization (IDPO) to attribute CTR gains to isolated elements. Our work is supported by AutoPP1M, the largest dataset specifically designed for product poster generation and optimization, which contains one million high-quality posters and feedback collected from over one million users. Experiments demonstrate that AutoPP achieves state-of-the-art results in both offline and online settings. Our code and dataset are publicly available at: https://github.com/JD-GenX/AutoPP