DraCo: Draft as CoT for Text-to-Image Preview and Rare Concept Generation

AI summary

Overview

• Research area: Computer vision, specifically text-to-image (T2I) generation with unified multimodal large language models (MLLMs) and chain-of-thought (CoT) reasoning. Published on arXiv (2512.05112v1 [cs.CV], 04 Dec 2025).
• Technical level: Intermediate. Readers need some familiarity with diffusion-style image generation, classifier-free guidance (CFG), vision-language models, and the idea of chain-of-thought reasoning.
• Scope: Introduces an interleaved text-and-image reasoning pipeline in which a low-resolution "draft" image serves as the chain-of-thought, plus a training dataset (DraCo-240K) and a specialized CFG strategy (DraCo-CFG) for correcting and upscaling that draft.

What This Paper Is About

Existing approaches that add reasoning to text-to-image generation either treat a unified MLLM purely as an image generator guided by an external reward model, or produce only abstract textual plans before synthesizing the image. Text alone is a coarse way to plan a dense visual output, and current models struggle with rare attribute combinations (the paper's example: "a white orange," which models fail on because "orange" is strongly bound to its typical color). DraCo's goal is to let the model generate a cheap low-resolution visual draft as its plan, use the model's own understanding ability to verify that draft against the prompt, and then correct and super-resolve it into the final image.

Key Contributions

1. Draft-as-CoT, an interleaved reasoning paradigm. A three-step pipeline — draft sketching, draft verification, and corrective refinement — that uses both visual and textual content as chain-of-thought for T2I generation, instead of text-only planning or an exterior reward model.
2. DraCo-240K, a curated training dataset. Over 240K interleaved reasoning instances covering three atomic correction capabilities: general correction, instance manipulation, and layout reorganization, each built with a dedicated automated data-synthesis pipeline.
3. DraCo-CFG, a classifier-free guidance strategy for interleaved reasoning. A single-round CFG formulation that separately emphasizes the draft's visual semantics and the prompt-plus-verification correction signal, replacing Bagel's sequential two-round CFG.
4. A training recipe and empirical validation. Supervised fine-tuning from Bagel with BCE loss on verifications and MSE on VAE tokens, two 5% dropout strategies to support DraCo-CFG, and evaluation on GenEval, Imagine-Bench, and GenEval++.

Main Findings

• GenEval gains. DraCo reaches an overall score of 0.86 on GenEval, compared with 0.78 for the Bagel baseline — an +8% improvement. It scores 1.00 on Single Obj., 0.99 on Two Obj., 0.81 on Counting, 0.91 on Colors, 0.70 on Position, and 0.76 on Color Attri., achieving the highest score in five of the six subtasks. The paper reports that DraCo is particularly effective at complex attribute combinations (the color-attribute subtask).
• Comparison against other CoT methods on GenEval. Bagel-Think scores 0.82 and T2I-R1 scores 0.79 overall; DraCo is reported as surpassing the text-CoT-based method by 4%. The exterior reward model PARM improves Show-o from 0.53 to 0.55 (2%), while DraCo's improvement starts from a stronger 0.78 baseline and is 6% greater.
• Imagine-Bench gains. DraCo reaches 7.11 overall, a +0.91 improvement over Bagel (6.20), and is 0.18 points above text-only reasoning (Bagel-Think, 6.93). Subtask scores: Attr. shift 6.40, Spatiotem. 7.30, Hybrid. 7.99, Multi-Obj. 7.20.
• GenEval++ gains. DraCo reaches 0.40 overall versus 0.37 for Bagel (+3%), while Bagel-Think scores 0.35. Subtask scores: Color 0.45, Count 0.53, Color/Count 0.28, Color/Pos 0.40, Pos/Count 0.28, Pos/Size 0.53, Multi-Count 0.38.
• Text-based planning can hurt. Bagel-Think underperforms vanilla Bagel on GenEval++, which the authors interpret as a potential limitation of text-based planning; DraCo performs better on all three benchmarks.
• DraCo-CFG beats the original CFG. On GenEval, DraCo-CFG exceeds Bagel's original CFG by 3% overall, and qualitative comparisons show improved clarity (for example in clock details) versus blurrier outputs from the original CFG.
• Draft VAE features are not needed. Including VAE features from the draft alongside ViT features gives 0.84 overall, which is 2% lower than the 0.86 obtained without them. Qualitatively, VAE features are linked to artifacts such as unnatural lighting on a horse and disconnected clock frame elements, attributed to over-adherence to the draft.
• 384×384 is the best draft resolution. Ablations give 0.76 at 128×128, 0.75 at 1024×1024, and 0.86 at 384×384 with DraCo-CFG. The authors explain that 128×128 is too small for planning and verification, while 1024×1024 inflates token length per training sample and reduces the number of samples seen in the same iterations.
• Baseline weaknesses motivated the dataset. A zero-shot pilot study found Bagel fails to follow correction instructions even when the verification is correct, and instead of respecting draft semantics it generates entirely new images. On Imagine-Bench, the base Bagel scores 6.20, and the paper notes it lacks the ability to understand spatial relationships in editing instructions.
• Not reported. Inference latency, wall-clock cost of the three-step pipeline, and any human evaluation of output quality are not reported in the paper content.

Methodology in Plain English

The method is built on Bagel, a unified MLLM with a ViT encoder for understanding, a VAE encoder for generation, and a Mixture-of-Transformer-Experts (MoT) with one branch for VAE tokens and another for ViT and text tokens; image generation uses Rectified Flow to produce VAE tokens.

DraCo runs in three steps:

1. Draft sketching. The prompt alone is used to produce a low-resolution draft, 384×384 rather than Bagel's usual 1024×1024. The smaller size is fast to generate but still shows all the semantics, including small objects.
2. Draft verification. The draft is encoded by the ViT and fed back into the model with the instruction to understand the image, compare it against the prompt, and describe what edits are needed. Only ViT features are used — VAE features are deliberately dropped so that low-level detail does not lock the model into the draft, allowing more substantial corrections.
3. Corrective refinement. The model produces the final high-resolution image, simultaneously upscaling and correcting the draft. Guidance is handled by DraCo-CFG, which combines an unconditional input, a draft-only input, and a fully-conditioned input as m(∅,∅,∅) + s_draft·(m(∅,vit,∅) − m(∅,∅,∅)) + s_text·(m(p,vit,v) − m(∅,vit,∅)), so the draft condition and the prompt-and-correction condition can be scaled independently. At evaluation, s_draft and s_text are set to 2 and 6 respectively.

Data construction. DraCo-240K is built in two stages. First, image pairs are collected as (draft, final) pairs: general correction reuses existing editing datasets (A-Image before edit, B-Image after); instance manipulation generates images with repeated objects, uses GroundingDINO for bounding boxes, masks selected objects with black rectangles, and has FLUX-Kontext inpaint the backgrounds, and uses GPT-4o to regenerate low-quality RefEdit edited images; layout reorganization uses GroundedSAM segmentation masks, randomly swaps object positions on a black background, and uses FLUX-Kontext to inpaint. Second, Qwen3-VL-235B-A22B receives the swapped image pair and produces a caption of the B-Image as the prompt and a verification explaining how a draft would be corrected. Samples are organized as "modification needed" (A-Image as draft) and "no modification needed" (B-Image as draft), plus text-to-image data at both low and high resolution. Prompts for instance manipulation and layout reorganization are cross-validated with GroundingDINO to check object numbers and positions.

Training. Full-parameter fine-tuning from Bagel for 16K steps with EMA weights, learning rate 2e-5, 2K warmup steps, the ViT encoder and its connector frozen, and a maximum concatenated sample length of 36K per GPU on 8 H800 GPUs. Because Bagel struggles to generate valid 384×384 images, a preliminary text-to-image fine-tuning stage runs for 14K steps, with half the images trained at 384×384 and half at 1024×1024. Losses are BCE on the verification text and MSE on the final-image VAE tokens, and two dropout strategies each with 5% probability support the CFG design.

Why This Matters

Impact on research. The paper reframes chain-of-thought for image generation: instead of text plans or external reward models, the intermediate reasoning step is itself an image at reduced cost, produced as a pre-plan rather than a post-hoc reflection. It also identifies that rare attribute combinations fail because training data underrepresents them, and shows a mechanism — preview, verify, correct — that mitigates this without forcing a perfect one-pass generation. The DraCo-240K pipelines and the DraCo-CFG formulation are reusable artifacts for other interleaved reasoning systems.

Real-world applications:

• Content creation and advertising, where specified color/object combinations ("a white orange," unusual product colorways) must be rendered faithfully from a prompt.
• Design and e-commerce mockups that need precise control over object count, attributes, and spatial layout in a scene.
• Iterative creative workflows where a quick low-resolution preview lets a user or system check alignment with intent before committing to a full-resolution render.
• Dataset and asset generation pipelines for downstream vision tasks, since the same draft-verify-correct loop produces paired images and instructions.

Industry relevance. The improvements land on benchmarks (GenEval, GenEval++, Imagine-Bench) used to compare commercial and open generative systems such as DALL-E 3, SDXL, SD3-Medium, FLUX.1-dev, and Janus-Pro-7B. Because DraCo is trained by fine-tuning an existing open unified MLLM and needs only 8 H800 GPUs, the approach is relatively accessible to teams already working with unified generation models, and the dataset construction pipelines rely on off-the-shelf tools (GroundingDINO, GroundedSAM, FLUX-Kontext, GPT-4o, Qwen3-VL).

Future Directions

• Extending the paradigm to other modalities. The authors state that the low-resolution draft design cannot be directly or optimally applied to videos, 3D assets, or scenes, and that even a low-resolution draft may be computationally expensive for video generation. A future draft would need to capture each modality's major difficulty — for example, temporal consistency in video.
• Human-in-the-loop curation and training. The paper notes that the role of humans in improving data curation and the training loop is not well studied; incorporating people could better align generation and correction methods.
• Generalizing correction capability. The pilot study showed Bagel's inherent editing ability was insufficient for correction scenarios such as layout adjustment, motivating the purpose-built dataset. Whether these atomic capabilities transfer beyond the three trained categories (general correction, instance manipulation, layout reorganization) is left open.
• Cost-performance trade-offs. The three-step pipeline's inference overhead relative to direct generation is not quantified in the paper content, leaving efficiency of draft-based reasoning as an open question.

Target Audience

Researchers and graduate students working on unified multimodal models, text-to-image generation, and reasoning-augmented generation; engineers building controllable image-generation products that need precise attribute, counting, and layout control; and practitioners interested in dataset construction pipelines that combine MLLMs, editing models, and segmentation models. Readers seeking evaluation-protocol or benchmark-design detail will also find the ablation structure (draft resolution, VAE-feature inclusion, CFG variant) directly useful.

Authors’ abstract

Recent unified multimodal large language models (MLLMs) have shown impressive capabilities, incorporating chain-of-thought (CoT) reasoning for enhanced text-to-image generation. However, existing approaches remain limited, either treating the model merely as a standalone generator or relying on abstract textual planning. To this end, we propose Draft-as-CoT (DraCo), a novel interleaved reasoning paradigm that fully leverages both textual and visual contents in CoT for better planning and verification. Our method first generates a low-resolution draft image as preview, providing more concrete and structural visual planning and guidance. Then, we employ the model's inherent understanding capability to verify potential semantic misalignments between the draft and input prompt, and performs refinement through selective corrections with super-resolution. In this way, our approach addresses two fundamental challenges: the coarse-grained nature of textual planning and the difficulty in generating rare attribute combinations. To support training, we curate DraCo-240K, aiming to enhance three atomic capabilities spanning general correction, instance manipulation, and layout reorganization. Supported by DraCo-CFG, a specialized classifier-free guidance (CFG) strategy for interleaved reasoning, DraCo achieves a tremendous increase on GenEval (+8%), Imagine-Bench (+0.91), and GenEval++ (+3%), significantly outperforming direct generation and other generation methods empowered by CoT.