From Prompting to Composing: A Spatial Canvas Interface for Poster Generation

AI summary

Overview

Research area: Computer vision — controllable image generation and graphic design, specifically poster generation via multimodal diffusion models.

Technical level: Intermediate. The interface concept (spatial bindings on a canvas) is intuitive, but the method assumes familiarity with diffusion transformers, LoRA adaptation, and reinforcement learning post-training for generative models.

Scope: This paper proposes a 2D "Spatial Canvas" interface for specifying poster layouts through four types of spatial bindings, and a model called Compo that is adapted from a pretrained image editing backbone to generate posters from that canvas.

What This Paper Is About

Text prompting asks users to squeeze two-dimensional composition intent — where things go, what they look like, what must stay unchanged — into a one-dimensional string of words, which is indirect and ambiguous for structured documents like posters. The authors replace that with a Spatial Canvas where users place semantic, identity, text, and pixel bindings directly at the locations they belong, plus accompanying Text Specifications that describe each bound element and the global background and style. Their goal is a model, Compo, that reads this structured canvas-and-text input and produces a high-quality poster that respects every binding.

Key Contributions

1. A Spatial Canvas Interface that decouples intent specification from visual generation, supporting four complementary binding types — Semantic Binding (a textual concept tied to a region), Identity Binding (a reference image tied to a region to preserve visual identity), Text Binding (an exact text string and its location), and Pixel Binding (visual content anchored for preservation) — freely combinable on one canvas alongside Text Specifications.

2. Compo, a poster generation model adapted from pretrained image editing models (LongCat-Image-Edit and Qwen-Image-Edit-2511) via LoRA rather than trained from scratch, supporting both direct inference (users explicitly build the canvas) and agentic mode (a high-level request is automatically converted into a planned Spatial Canvas and Text Specifications by a GPT-5.6-based agent).

3. A scalable training data generation pipeline and the Compo-200K dataset (roughly 200K samples), which automatically constructs supervision for each binding type and their combinations using instance segmentation, visual asset processing, and text recognition, avoiding prohibitively expensive manual canvas–poster annotation.

4. A benchmark and evaluation protocol of 500 samples (no overlap with training data) that measures adherence to each binding type individually and their joint composition, rather than only overall image quality.

Main Findings

• Compo outperforms both generalist and specialist baselines on nearly all metrics. Compo-LongCat reaches an overall average score of 0.752 and overall IoU of 0.885; Compo-Qwen reaches 0.751 and 0.886. The strongest general text-and-image-to-image baseline, HiDream-O1-Image, reaches 0.504 average score and 0.547 overall IoU. Among dedicated poster systems, PosterMaker reaches 0.466/0.628, PosterOmni 0.447/0.463, and CreatiDesign 0.426/0.561.

• Text Binding shows the largest gap. Compo-LongCat achieves 0.970 NED and 0.940 sentence accuracy with 0.944 IoU; Compo-Qwen achieves 0.940 NED, 0.868 sentence accuracy, and 0.934 IoU. The best baseline sentence accuracy is PosterMaker's 0.107 at 0.397 NED, and CreatiDesign reaches 0.329 sentence accuracy at 0.515 NED.

• Identity preservation improves substantially over baselines. Compo-Qwen records 0.910 CLIP-I and 0.781 M-DINO under Identity Binding, versus 0.776/0.642 for HiDream-O1-Image; PosterMaker is far weaker on identity (0.329 CLIP-I, 0.157 M-DINO) despite its strength on pixel preservation.

• Pixel Binding favors the dedicated baseline, but Compo is competitive. PosterMaker leads this category with 0.708 DINO-I, 0.748 CLIP-I, and 0.760 IoU, while Compo-LongCat records 0.735 DINO-I, 0.776 CLIP-I, 0.642 M-DINO, 0.793 SSIM, and 0.781 IoU. Dedicated poster systems like CreatiDesign (0.210 CLIP-I) and PosterOmni (0.675 CLIP-I) are much weaker on identity.

• All three reward components contribute in RL post-training. On Compo-LongCat, the full reward yields 0.752 average score; removing the text reward drops it to 0.746, removing the instance similarity reward to 0.736, removing the layout reward to 0.750, and removing all rewards to 0.725. Text-binding metrics suffer most without the text reward (sentence accuracy falls from 0.940 to 0.778), and IoU-related metrics fall without the layout reward.

• A sentence-accuracy weight of 0.75 balances text correctness best. Weight 0.75 gives 0.940 sentence accuracy, 0.970 NED, and 0.944 IoU, versus 0.916 sentence accuracy at weight 0.0 and 0.938 at weight 1.0 (where the reward becomes overly sparse early in training).

• Fine-tuning both MMDiT branches is best. Both branches achieve 0.725 overall average score versus 0.721 for image branch only and 0.722 for text branch only; the two branches play complementary roles for structured VL tokens and canvas tokens.

• Agentic mode removes the need for manual canvas construction. A GPT-5.6-based agent performs element extraction, layout planning, binding selection, and image search to produce the planned canvas automatically.

Methodology in Plain English

The authors start from the observation that posters have structure: subjects, text, logos, and graphics live at intentional locations and often must preserve a specific identity. Instead of encoding all of that in a prompt, they let users draw it. A black canvas matches the target poster's resolution; bounding boxes, pasted reference images, and text identifiers are literally rendered onto the canvas as visual conditions. Four binding types cover the space of constraints — semantic (a class name in a box), identity (a reference image to preserve), text (an exact string), and pixel (content pasted for preservation) — while a separate text specification records per-element descriptions and global background/style attributes.

To train a model on this interface, they needed paired data, which does not exist. So they built a pipeline: large language models (GPT-5.5, Claude Opus 4.6, Gemini 3.5 Flash) first curate a content repository of about 1,000 instances across 12 poster themes and a style repository of around 250 fine-grained style descriptions, then plan a structured specification and rewrite it into a natural prompt while preserving all planned information. Text-to-image models (Qwen-Image-2512, Z-Image, ERNIE-Image) render each prompt at 1024² resolution on roughly 32 NVIDIA A100 80GB GPUs. Working backwards from each rendered poster, SAM-3 segments instances, FLUX.2-klein-9B re-lights or re-poses them into canonical forms, and Chandra-OCR-2 recognizes the rendered text and its location — the OCR output, not the original plan, becomes ground truth. This yields roughly 200K samples (Compo-200K).

Compo itself is a pretrained MMDiT diffusion transformer with frozen text (Qwen2.5-VL) and VAE components. LoRA parameters attached to the MMDiT are the only thing trained. Training has two stages: supervised training on Compo-200K, then reinforcement learning with DiffusionNFT using three binding-aware rewards — a text reward combining sentence accuracy and normalized edit distance, an instance similarity reward combining DINO-I, CLIP-I, and CLIP-T, and a layout reward based on IoU between intended and generated regions. Sentence accuracy is aggregated with a logical AND across all bound text elements, so an image scores 1 only if every string is rendered correctly; other metrics use arithmetic averaging.

Evaluation uses the same pipeline to build a 500-sample benchmark with 4–9 instances per sample (deliberately more than the 3–8 used in training). A unified region-matching protocol matches each detected instance or text region to the ground-truth region with the highest IoU; unmatched regions score zero across all metrics.

Why This Matters

Impact on research. The paper reframes controllable generation as binding generation constraints to spatial locations on a shared canvas, arguing that layout control, reference-driven generation, text rendering, and image editing are all special cases of that principle. It also shows a lightweight adaptation route — LoRA on a frozen editing backbone plus automatically generated supervision — for teaching an existing model an entirely new conditioning interface, and it contributes both a dataset and an evaluation protocol where the field previously lacked them for structured graphic design.

Real-world applications.

• Marketing and advertising: generating campaign posters where a brand logo, product shot, or spokesperson identity must be preserved exactly while layout and copy change.
• E-commerce and retail: producing promotional graphics at scale where product images (pixel binding) and price/offer text (text binding) must be rendered correctly.
• Event and public service communication: assembling notices and announcements where specific wording and placement are non-negotiable.
• Design tooling: structured canvas representations as an intermediate layer that lets a design tool or an agent drive a generative model instead of a text box.

Industry relevance. Because Compo is built by adapting existing open-source editing models rather than training a new generator, its approach is practical for teams that already have an editing backbone in production. The agentic mode is directly relevant to consumer-facing design assistants, where users type a request and expect a finished layout. The benchmark also gives vendors a way to compare compositional controllability rather than only aesthetic quality.

Future Directions

• Extending beyond posters. The interface principle is general to structured graphic design, but the paper only demonstrates it on posters; whether it transfers to web pages, slides, or packaging is not reported.
• Reducing dependence on automated data and LLM planning. The whole pipeline leans on proprietary models for planning, rendering, segmentation, and OCR; error propagation from these stages into training supervision is not quantified in the available content.
• Strengthening pixel-level preservation. Compo trails PosterMaker on some Pixel Binding metrics (DINO-I 0.735 vs. 0.708 favors Compo, but PosterMaker leads on IoU 0.760 vs 0.781 in Compo's favor and on CLIP-I 0.748 vs 0.776), suggesting the strictest preservation case still has room for improvement. (Note: the paper's Pixel Binding comparison is mixed rather than a clean win.) Further error analysis is not reported in the available content.
• Better evaluation of the agentic path. The paper describes agentic mode's components — element extraction, layout planning, binding selection, image search — but end-to-end quality of agent-planned canvases versus human-built ones is not reported in the provided content, and the appendices on this (Appendix B.3, C.1, C.2) are referenced but truncated here.

Target Audience

Researchers and practitioners in generative modeling, controllable image synthesis, and computational design who want to move beyond prompt-only interfaces for structured visual outputs. It is especially relevant to engineers building design assistants or ad-generation systems, and to researchers studying layout-conditioned generation, reference-guided generation, or reinforcement learning post-training of diffusion models. Readers primarily interested in the interface idea can follow it without deep diffusion expertise, but the training and ablation sections require comfort with MMDiT architectures, LoRA, and reward-based fine-tuning.

Authors’ abstract

Text prompting is an indirect interface for poster generation, requiring users to encode inherently two-dimensional composition intent into a one-dimensional sequence of words. We introduce a Spatial Canvas Interface that enables users to directly compose generation intent in space through four complementary binding types: semantic, identity, text, and pixel, together with Text Specifications for individual elements and global appearance. Based on this interface, we develop Compo, a poster generation model adapted from a pretrained image editing model to understand Spatial Canvas inputs and Text Specifications. Compo supports both direct inference, where users explicitly construct the canvas, and agentic mode, where a high-level request is automatically translated into a planned Spatial Canvas. To train Compo, we develop a scalable pipeline that automatically constructs supervision data for different binding types and their combinations, enabling efficient adaptation without training a specialized poster generator from scratch. We further introduce a benchmark that evaluates adherence to individual binding types and their joint composition. Experiments show that Compo achieves stronger compositional controllability than both general-purpose image generation models and dedicated poster generation systems while maintaining high visual quality. By decoupling intent specification from visual generation, our work shifts poster generation from prompting toward composing.