Thinking-while-Generating: Interleaving Textual Reasoning throughout Visual Generation

AI summary

Overview

Research area: Computer vision, specifically text-to-image generation and multimodal reasoning (interleaving textual chain-of-thought with visual synthesis).

Technical level: Advanced. The paper assumes familiarity with autoregressive image generators, unified understanding–generation models (ULMs), supervised fine-tuning, reinforcement learning (GRPO), and T2I composition benchmarks.

Scope (1 sentence): This paper introduces Thinking-while-Generating (TwiG), the first framework that interleaves textual reasoning throughout the visual generation process rather than before or after it, and evaluates three strategies — zero-shot prompting, supervised fine-tuning on a purpose-built dataset, and a customized GRPO reinforcement learning scheme — on top of Janus-Pro-7B under an autoregressive text-to-image setting.

What This Paper Is About

Existing "reasoning for image generation" methods insert chain-of-thought either as pre-planning (a fixed plan before synthesis) or as post-refinement (a critique-and-revise pass after the image is finished). Both leave the generation process itself reasoning-free: the plan cannot adapt mid-course, and post-hoc correction is coarse and costly. Inspired by image-text interleaved reasoning in multimodal understanding models, the authors invert the modality flow and ask whether textual thoughts can be woven into an unfolding image, providing on-the-fly guidance for upcoming regions and reflection on already-drawn regions within a single generation trajectory.

Key Contributions

1. A new paradigm, Thinking-while-Generating (TwiG). The first interleaved framework where textual reasoning co-evolves with visual generation. It is decomposed into three schemes: When to Think (scheduling reasoning points), What to Say (producing a local sub-caption that steers the next visual region), and How to Refine (a scored, region-level reflection with optional local correction).

2. A single-trajectory, image-to-image-free mechanism. By extending the textual pre-context from {τ_j}_{j<k} to {τ_j}_{j≤k} and re-generating only a local visual region, the framework preserves one autoregressive T2I trajectory — no fresh pass, no full re-generation, and no requirement that the generator itself have image-to-image ability.

3. A curated dataset, TwiG-50K. Approximately 50K samples built with commercial models, split into roughly 17K "what to say" (thinking) samples, roughly 17K "how to refine" (reflection) samples, and roughly 16K generation samples, covering nine subtasks (three thinking, three reflection, three generation). The thinking/refinement data derives from 5.5K prompts from the T2I-CompBench training split.

4. A customized RL strategy, TwiG-GRPO. A GRPO variant where a single reward computed from the final image and input prompt is shared across all thinking, generation, and reflection passes, plus a four-model reward ensemble (HPS v2, GroundingDINO, GIT, fine-tuned ORM).

Main Findings

• Zero-shot prompting already works surprisingly well. With hand-designed interleave-aware prompts, TwiG-ZS beats Janus-Pro-7B on T2I-CompBench by +9.52 (Color, 73.11 vs. 63.59), +6.27 (Shape, 41.55 vs. 35.28), +15.41 (Texture, 64.77 vs. 49.36), +1.37 (Spatial, 21.98 vs. 20.61), +0.05 (Non-Spatial, 30.90 vs. 30.85), and +12.57 (Complex, 48.16 vs. 35.59).

• Interleaving beats thinking only before or only after. (Ablation a) Thinking-while-Gen. scores 73.11/41.55/64.77/21.98/30.90/48.16 vs. Think-before-Gen. 65.12/36.20/51.05/20.88/30.82/41.75 and Think-after-Gen. 64.72/37.95/50.62/21.05/30.87/42.28.

• K = 3 reasoning steps is optimal. (Ablation b) K=2 gives 72.79/42.26/64.64/21.97/30.89/49.71; K=3 gives 73.11/41.55/64.77/21.98/30.90/48.16; K=4 gives 72.95/41.90/64.70/22.03/31.10/48.90.

• Uniform spacing beats adaptive spacing. (Ablation c) Uniform scores 73.11/41.55/64.77/21.98/30.90/48.16 vs. Adaptive 72.43/40.88/63.92/21.67/30.88/47.39 — the authors report current ULMs cannot reliably produce well-structured adaptive schedules.

• One reflection round helps; a second does not. (Ablation d) Without reflection: 73.11/41.55/64.77/21.98/30.90/48.16. One round: 73.90/46.02/66.10/24.50/30.81/51.97. Two rounds: 73.68/45.72/66.02/24.42/30.88/51.65.

• SFT on TwiG-50K provides modest, consistent gains. TwiG-SFT scores 74.58/52.42/67.95/27.02/31.24/53.41, improving over TwiG-ZS by +1.47/+10.87/+3.18/+5.04/+0.34/+5.25. Balanced thinking-and-generation data ("Think-Gen-equal") is best; adding reflection data hurts (Reflect-lite 72.76/49.75/65.93/26.36/30.92/51.17; Reflect-heavy 71.88/48.98/65.05/25.62/30.84/50.27).

• SFT also improves stability. Across 5 random seeds, TwiG-SFT standard deviations (0.65/0.59/0.61/0.40/0.36/0.80) are lower than TwiG-ZS (0.82/0.70/0.76/0.45/0.38/0.91).

• RL delivers the largest jump. TwiG-RL scores 82.49/61.28/73.19/34.06/31.99/54.45, exceeding TwiG-SFT by +7.91, +8.86, +5.24, +7.04, +0.75, and +1.04 respectively.

• Joint reinforcement is best. (RL ablation a) TwiG-GRPO (82.49/61.28/73.19/34.06/31.99/54.45) outperforms ULM_g-GRPO (80.12/59.87/72.01/32.47/31.30/54.02) and ULM_u-GRPO (78.36/57.94/70.68/30.93/31.27/53.76).

• Layered reward ensembles steadily improve results. (RL ablation b) Human Preference 79.83/60.97/71.35/20.68/30.53/52.87; + Object Grounding 80.44/60.01/73.79/25.84/31.15/54.03; ++ VQA Consistency 80.87/59.29/74.26/30.05/31.41/53.64; +++ LMM Alignment 82.49/61.28/73.19/34.06/31.99/54.45.

• On T2I-CompBench++, TwiG-RL is highly competitive. The paper reports TwiG-RL at 82.49 (Color), 61.28 (Shape), 73.19 (Texture), 34.06, 38.87, 31.99, 61.93, and 53.56 across the benchmark's columns, alongside comparisons against Show-o, SD-XL-base-1.0, Attend-and-Excite, PixArt-α, GoT, Show-o + PARM, FLUX.1, Emu3, and T2I-R1.

• Relation to concurrent work. The authors note that IRG and Uni-CoT also "interleave" reasoning with generation but treat synthesis as a monolithic block — effectively think-before plus think-after — and therefore do not reason within the generative process itself.

Methodology in Plain English

The authors build on a unified understanding–generation model (Janus-Pro-7B, an autoregressive ULM) and treat image synthesis as a sequence of locally reasoned steps.

1. Decide when to think. Given a text prompt, the understanding pass produces a schedule — how many reasoning checkpoints to use (K) and which visual regions each checkpoint covers. In this study the schedule is static with K = 3, motivated by the heuristic that images decompose into upper background, central content, and lower background. (Data for adaptive scheduling is deliberately not collected; the paper reserves it for future work.)

2. Decide what to say. At each checkpoint the understanding pass writes a short sub-caption for the next region, conditioned on the prompt, prior thoughts, and prior visual content. The generation pass then draws that region conditioned on all thoughts so far. Crucially, prior visual content is never fed back as an image — only the textual pre-context grows — so the model only needs text-to-image ability and stays on a single generation trajectory.

3. Decide how to refine. After a region is drawn, the understanding pass emits a reflection tuple: an integer critic score in [0, 100] and a possible revised sub-caption. If the score exceeds a threshold, generation continues; otherwise only that local region is regenerated using the revised caption. The paper's prompts specify five scoring criteria: color accuracy, object completeness, detail richness, spatial relationships, and visual coherence.

4. Three training routes are compared. (i) Zero-shot: only carefully

Authors’ abstract

Recent advances in visual generation have increasingly explored the integration of reasoning capabilities. They incorporate textual reasoning, i.e., think, either before (as pre-planning) or after (as post-refinement) the generation process, yet they lack on-the-fly multimodal interaction during the generation itself. In this preliminary study, we introduce Thinking-while-Generating (TwiG), the first interleaved framework that enables co-evolving textual reasoning throughout the visual generation process. As visual content is progressively generating, textual reasoning is interleaved to both guide upcoming local regions and reflect on previously synthesized ones. This dynamic interplay produces more context-aware and semantically rich visual outputs. To unveil the potential of this framework, we investigate three candidate strategies, zero-shot prompting, supervised fine-tuning (SFT) on our curated TwiG-50K dataset, and reinforcement learning (RL) via a customized TwiG-GRPO strategy, each offering unique insights into the dynamics of interleaved reasoning. We hope this work inspires further research into interleaving textual reasoning for enhanced visual generation. Code will be released at: https://github.com/ZiyuGuo99/Thinking-while-Generating.