MOSS-Image Technical Report

AI summary

Overview

Research area: Computer vision / text-to-image generation, specifically open-weight generative systems for text-rich, layout-heavy graphics (posters, infographics, user interfaces).

Technical level: Advanced — the paper assumes familiarity with diffusion transformers, flow matching, RoPE positional encodings, and human-preference evaluation (Bradley–Terry / Elo).

Scope in one sentence: The report describes an open 5B diffusion transformer pretrained from scratch plus an agentic prompt enhancer, and two new benchmarks (LLTB and RLA) for long-form text rendering and end-to-end layout-rich generation.

What This Paper Is About

Closed-source image generators can produce information-dense graphics (infographics, UIs, knowledge cards), while open-weight models still struggle with dense text and intricate layouts. The authors argue this gap has two coupled causes: rendering (faithfully drawing long text and following spatial constraints) and specification (turning a five-word request like "a poster about X" into actual content, hierarchy, and layout). MOSS-Image is their attempt to solve both at once — a purpose-built generator plus an agent that writes detailed content and layout plans before any pixels are generated.

Key Contributions

1. A pretrained foundation model for information-dense graphics. A 5B single-stream diffusion transformer (DiT) trained from scratch, with a documented recipe: broad data retention (no extra heuristic filtering), multi-granularity and structure-aware text supervision, and progressive-resolution training. Natural-language descriptions and structured layout JSON are learned through a single text-conditioning pathway, so the model follows either form.

2. An agentic end-to-end generation system. An intention-aware prompt enhancer ("Layout PE," built on GPT-6 with modular task instructions) that routes each request to task-specific guidance, retrieves knowledge and visual references, and emits an executable JSON layout — literal text, elements, bounding boxes, typography, and spatial relationships — that the DiT renders directly.

3. Two new benchmarks. LongLongTextBench (LLTB): 240 literary-text rendering tasks (120 English, 120 Chinese, three body-length bins). RealLayoutArena (RLA): 61 tasks built from design references (20 posters, 20 infographics, 21 user interfaces), scored by blind pairwise human comparison on an Elo scale.

4. Systematic ablations separating specification from rendering. Prompt-strategy comparisons and a cross-backbone comparison under shared enhanced prompts, so the enhancer's contribution and the generator's contribution can be read separately.

Main Findings

• State of the art among open-weight models on LLTB: MOSS-Image reaches a Mean-6 score of 96.3, versus 64.9 for Qwen Image 2512 — a 31.4 percentage-point gap. On the longest English and Chinese conditions it scores 97.1 and 92.4 respectively; the authors note recall is lower for the longest Chinese texts.

• Strong layout control via text alone: mean IoU over detected target objects (mIoU_det) of 0.88 and Layout AUC of 0.70, compared with 0.44 and 0.20 for Ideogram 4 — leading all evaluated models on both metrics.

• Second place end-to-end on RLA: 1698.70 Elo (95% CI 1655.96–1741.43), behind GPT Image 2 (1844.35) and ahead of Seedream 5.0 Pro (1661.16), Qwen Image 3.0 Pro with PE (1614.84), and Nano Banana Pro (1590.19) across nine systems compared.

• First place when the generator is isolated: under shared enhanced natural-language prompts across nine generators, the MOSS-Image backbone scores 1807.34 Elo (95% CI 1727.44–1887.24), ahead of ERNIE Image (1748.51) and SenseNova-U1 (1689.23).

• Detailed specifications beat short prompts: short requests score 1081.70 Elo; detailed prompts 1509.21; direct expansion 1541.45; JSON layout the highest at 1622.87; natural-language expression of the same plan 1506.20.

• Filtering data hurts in their ablation: adding a heuristic quality filter retained 513,110,382 of 1,183,622,445 records (43.35%) but degraded GenEval, OneIG, and FDR6-LAION, with the OneIG text score collapsing from 0.5243 to 0.1180.

• Optimizer choice matters: in a 1B ablation, hybrid Muon–AdamW reached denoising losses of 0.41 and 0.403 with 33.2% and 28.6% fewer steps than AdamW.

• LongTextBench is saturating: collected results cluster between 96.05 and 98.85, with targets averaging only 120.3 characters — hence the shift to LLTB.

Methodology in Plain English

The generator. They train a 5B-parameter single-stream diffusion transformer from scratch. Text comes from a frozen Qwen3.5-9B encoder (final-layer hidden states, dimension 4096, projected to 3072 by a linear adapter plus two text-only Transformer blocks); images come from a frozen FLUX.2 VAE (32 latent channels, 8× downsampling), with 2×2 latent patches giving an effective 16-pixel stride. The joint backbone has 42 blocks, hidden size 3072, 24 attention heads of 128 dimensions, and FFN hidden size 8192, using joint self-attention, shared modulation, and SiTU-GLU feed-forward layers. Positional encoding is a 4D RoPE over reference index, sequence position, width, and height; with no reference inputs all tokens use reference index 0, leaving the leading 32 of 128 head dimensions unrotated (partial RoPE) and reserving that subspace for future reference-conditioned generation.

The data recipe. Starting from ReLAION, OmniCorpus, DataComp and other public image-text datasets, they deduplicate with SSCD and strip AI-generated images, shrinking roughly 5 billion to 3 billion images. They deliberately apply no further filtering — watermarks, blurry faces, and compression artifacts are described in the captions rather than used as exclusion criteria. Images are recaptioned with gemma4 at three granularities (short sentence, medium paragraph, detailed description), and a subset gets structured JSON annotations with global description, localized elements, bounding boxes, categories, and verbatim text transcription. Line breaks are preserved in that transcription, because line breaks encode how text is arranged, not just what it says. During training, original and recaptioned captions are sampled at 20:80, with uniform 1:1:1 selection among short/medium/long when all three exist.

Training. Three progressive stages raise resolution and text-context capacity, with a final-stage cap of 1024² pixel area, dimensions aligned to multiples of 16, and a 4096-token text limit. The loss is flow matching with per-token normalization in the packed final stage, and noise sampling uses a staged truncated logit-normal schedule (shift 3 early, then shift 1) with a 1% uniform mixture. Optimization is hybrid Muon–AdamW at learning rate 5×10⁻⁵, weight decay 0.01, gradient clipping 1.0, EMA 0.9999, and 2000 warmup steps; the released Stage 3 checkpoint is taken at step 310,000 with EMA weights.

The enhancer. At inference, GPT-6 selects task modules by medium, pulls in knowledge and reference images as planning aids (they do not condition the DiT), and writes a validated JSON specification — literal text, elements, boxes, typography, spatial relations — that goes through the same text-conditioning interface the model was trained on. Line breaks and zero-centered absolute bounding-box coordinates turn out to generalize better than relative [0, 1000] coordinates for unseen resolutions.

Evaluation. Base-model rendering is measured on LongTextBench, LLTB, and 7Bench without the enhancer; end-to-end quality is measured on RLA with blind human pairwise comparisons fitted to an Elo scale.

Why This Matters

Research impact. The paper reframes text-rich generation as a two-part problem — specification and rendering — and provides the evaluation infrastructure to study them separately. LLTB (1,339.8 average characters per target, 321,545 characters total, roughly 11.1× and 8.4× LongTextBench, whose shortest target of 444 characters already exceeds LongTextBench's longest of 482) gives the field a non-saturated long-text benchmark. The release of model, training recipe, and evaluation framework under an open license is a meaningful counterweight to closed systems.

Real-world applications:

• Automated poster and marketing-graphic design from a one-line brief.
• Infographics and data-rich knowledge cards where text must be legible and complete.
• User-interface mockups with coherent element placement.
• Bilingual (English/Chinese) signage, menus, and page-length document layout.

Industry relevance. The "annotate rather than filter" finding and the Muon convergence result are directly reusable by anyone training a large diffusion model; the JSON-layout conditioning route shows that an open backbone can be driven by an external planner agent without retraining, which fits commercial pipelines that want to swap planners independently of the renderer.

Future Directions

1. Image-conditioned generation and editing. The authors state the current backbone does not condition on reference images or support editing, and that extending to image-conditioned generation and iterative design refinement is future work under their current compute budget. The reserved reference-index RoPE axis is an explicit hook for this.

2. De-risking the planner. The enhancer depends on external language models whose aesthetic judgment, reference interpretation, and factual errors can degrade the specification even when rendering is faithful — an open question is how much of that can be internalized or verified.

3. Closing the remaining text gap. Chinese recall on the longest texts trails English (92.4 versus 97.1), and training captions are predominantly English with no dedicated Chinese-caption annotation, so where the bilingual gap originates is unresolved.

4. System-level versus generator-level ranking. The backbone ranks first under shared enhanced prompts but the full system ranks second end-to-end, so identifying which upstream planner capabilities account for the difference is a natural next experiment. (The paper reports the rankings but does not attribute the gap.)

Target Audience

Researchers and engineers training or fine-tuning text-to-image diffusion models, especially those working on text rendering, layout control, or data-recipe design; teams building agent-plus-generator pipelines; and benchmark designers interested in human-preference evaluation of complex graphic design. Readers need comfort with diffusion transformer internals to follow Section 3 and the appendices, though the system-level story in Sections 2, 4, and 5 is accessible to product and design-technology readers.

Note: the paper does not report total training compute, the full parameter count of the frozen text encoder beyond its Qwen3.5-9B designation, or a per-language RLA breakdown.

Authors’ abstract

Recent closed-source image generation systems can produce diverse visual content from simple compositions to information-dense graphics, yet open-weight text-to-image models still struggle with dense text and intricate layouts. To narrow this gap, we introduce MOSS-Image, an open generation system that combines high-fidelity visual synthesis with intent-aware composition. First, we pretrain a 5B single-stream diffusion transformer (DiT) from scratch with a carefully designed recipe, enabling bilingual long-form text rendering and precise layout control. Second, we develop an intention-aware agentic prompt enhancement (PE) module that translates diverse user requests into detailed content and layout specifications. Together, they enable MOSS-Image to generate text-rich and layout-coherent images. We further introduce LongLongTextBench (LLTB) for evaluating long-form English and Chinese text rendering, and RealLayoutArena (RLA) for end-to-end layout-rich generation. MOSS-Image achieves state-of-the-art performance among open-weight models on LLTB, with an average text coverage of 96.3%. On RLA, MOSS-Image ranks second in the end-to-end system comparison, while its backbone ranks first among generators evaluated with shared enhanced prompts. We will release the model, training recipe, and evaluation framework to support further research and promote the text-to-image community.