ShowTable: Unlocking Creative Table Visualization with Collaborative Reflection and Refinement

AI summary

Overview

Research area: Computer vision / multimodal generation — specifically the intersection of multimodal large language models (MLLMs), diffusion-based image generation, and automated data visualization.

Technical level: Advanced. The paper assumes familiarity with diffusion models, MLLM reasoning, reinforcement learning (GRPO), and reward modeling.

Scope: The paper defines a new task (creative table visualization), proposes a self-correcting MLLM-plus-diffusion pipeline called ShowTable, releases three automated training-data construction pipelines, and introduces TableVisBench, a benchmark of 800 table instances evaluated along five dimensions.

What This Paper Is About

Existing image generation and "unified" models handle general image synthesis well, but fail at tasks that require deep reasoning, planning, and precise data-to-visual mapping. The authors introduce creative table visualization: given a table, the model must produce an infographic that is both aesthetically designed and faithful to every data point. Their goal is a system that reasons about layout and design while rigorously preserving quantitative data, since an incorrect bar height, pie-slice angle, or axis label makes such an output useless.

Key Contributions

1. A new task: creative table visualization, which requires detailed reasoning plus precise alignment between tabular data and generated visual elements, framed as a challenge to existing unified models.
2. Data: three automated data construction pipelines producing (a) 30K rewriting SFT pairs, (b) 5K refinement samples, and (c) 30K preference pairs for reward-model training, all built from 30K collected table-image pairs.
3. A method: ShowTable, a progressive self-correcting pipeline in which an MLLM acts as central orchestrator (rewriting, reflection) and a diffusion model acts as executor (generation, refinement).
4. A benchmark: TableVisBench, 800 challenging table instances across five evaluation dimensions (data accuracy, text rendering, relative relationship, additional information accuracy, aesthetic quality), with manual verification and correction of every sample.

Main Findings

• Base models fail badly at the task. Flux scores 29.3, Bagel 10.1, Blip3o-Next 10.8, UniWorld-V1 14.8, OmniGen2 14.4, and Qwen-Image 44.3 on TableVisBench. Bagel and Blip3o-Next score 0.1 and 0.4 respectively on Data Accuracy, which the authors describe as a fundamental failure to translate table data into visual components.
• The rewriting module is critical for layout and logic. Adding it to Qwen-Image raises Relative Relationship from 26.1 to 50.1 (overall score 44.3 to 54.3). For Bagel, the full pipeline improves the final score by +22.6; for Blip3o-Next, Data Accuracy rises from 0.5 to 21.3 and Text Rendering from 14.5 to 63.9.
• The full pipeline is best in all cases. Qwen-Image reaches 54.9 overall (up from 44.3, a +10.6 gain), with Data Accuracy 52.4 and Relative Relationship 54.3. The largest relative gains appear on weaker base models.
• Fine-tuned rewriting beats general-purpose LLMs. The Qwen3-8B-based rewrite module (Qwen3-8B*) achieves the best overall score among rewriting variants (54.3) and the best Data Accuracy (51.2), versus 30.6–40.8 for general LLMs. It even slightly exceeds the Reference-Caption upper bound on Data Accuracy (51.2 vs. 50.3).
• Refinement capability is the pipeline bottleneck. With untrained Qwen-Image-Edit-2509, scores degrade each round (54.3 → 51.8 → 50.1 → 49.4 across 0–3 rounds). With the RL-trained Qwen-Image-Edit-2509*, scores improve (53.7 → 54.8 → 54.9). Wan2.5-I2I-Preview improves continuously (61.3 → 62.8 → 63.4).
• Training the refiner yields measurable gains. Qwen-Image-Edit-2509* improves overall score from 49.4 to 54.9 (+5.5), with Data Accuracy +9.8 and Relative Relationship +9.3 over its base model.
• Stronger reflection models help. With the trained refiner, GPT-5 as the reflection module gives the best overall score (54.9); with the base Qwen-Image-Edit, Gemini-2.5-pro is best (51.2).
• Human/ground-truth reference images score highly on the benchmark. Reference images achieve Data Accuracy 97.7, Text Rendering 99.5, Relative Relationship 86.4, Additional Information Accuracy 96.6, Aesthetic Quality 4.2, and overall 84.4, which the authors use to validate that their metrics align with human-annotated quality.
• Code-based chart generation wins on correctness but loses on aesthetics. A Gemini-2.5-pro code baseline scores 79.4 overall versus 67.9 for the ShowTable pipeline, and much higher on Data Accuracy (83.8 vs. 69.6), but lower on Aesthetic Quality (4.1 vs. 4.9) and produces artifacts such as overlapping text.

Methodology in Plain English

The pipeline runs four stages in a loop:

1. Rewriting: An MLLM reads the raw markdown table and produces a detailed descriptive prompt, reasoning about how to present the data and what layout to use. Without this step, generators tend to re-render the table text rather than visualize it. The authors fine-tune Qwen3-8B for this using 30K SFT pairs of the form {table, rationale} → {description}, generating the descriptions and rationales with Gemini-2.5-pro.
2. Generation: A pretrained text-to-image model turns the rewritten prompt into an initial image, which typically captures overall layout but contains errors in data correspondence, axis labels, or text.
3. Reflection: An MLLM compares the generated image against the original table, identifies inconsistencies, and writes precise editing instructions. The default reflection model is GPT-5-2025-08-07.
4. Refinement: An image editing model applies the instructions. The maximum number of self-correction rounds is 3, with optional early termination if reflection judges the image satisfactory.

To make refinement work, the authors train a reward model (a fine-tuned Qwen2.5-VL-3B using Bradley-Terry loss on 30K preference pairs, with the scalar score computed from the probabilities of digits 0–9 in the output logits) and then apply reinforcement learning with GRPO — following Flow-GRPO — on 5K filtered samples, combining their reward model with an existing aesthetic reward, ImageReward. The refinement model is a distilled 8-step version of Qwen-Image-Edit-2509, chosen to accelerate RL training.

Training data was collected from SlideVQA, OpenImages, and Cambrian-10M. Images below 200×200 or lacking text (detected by PaddleOCR) were discarded. Gemini-2.5-pro and GPT-5 independently filtered for statistical-data images and annotated tables in markdown; only mutually consistent, mutually approved annotations were kept, yielding 30K table-image pairs. Refinement data was filtered by having the base editor produce five candidates per sample and discarding cases judged by GPT-5 as all-worse or all-better than the original, isolating samples that are neither too hard nor too easy.

The rewriting module was trained with LLaMA-Factory for 3 epochs, total batch size 256, learning rate 1e-5, and cosine learning rate decay.

Evaluation uses five dimensions: Data Accuracy, Text Rendering, Relative Relationship, and Additional Information Accuracy (each 0–100), plus Aesthetic Quality (0–10). The first four are scored by having an MLLM identify and quantify specific errors rather than give subjective scores; the final score is computed as (DA + TR + RR + AA + 10 × AQ) / 5.

Why This Matters

Impact on research: The paper reframes table visualization as a generative, design-oriented problem rather than a rule-based rendering problem. It argues that generation models have a higher ceiling for flexibility and aesthetics than code or template approaches, and that equipping them with data fidelity is a frontier for general visual synthesis. The three data construction pipelines and the 800-instance benchmark give the community reusable infrastructure, and the finding that refinement quality — not pipeline structure — limits self-correcting loops is a generalizable lesson for other iterative generation systems.

Real-world applications:

• Poster design and graphic design workflows that need data-driven content.
• Automatic slide generation from tabular business or scientific data.
• Scientific communication and data-driven reporting, where accurate figures must also look polished.
• Business intelligence dashboards where aesthetic appeal matters alongside numerical correctness.

Industry relevance: The modular design lets practitioners swap in different MLLMs and diffusion models: results are shown for Flux, Bagel, Blip3o-Next, UniWorld-V1, OmniGen2, and Qwen-Image as generators, and for Qwen3-VL-235B, Gemini-2.5-pro, and GPT-5 as reflection modules. The authors also show that RL-based training can substantially boost an open-source editing model (Qwen-Image-Edit-2509, +5.5 overall), offering a path to customized refinement solutions without relying on closed models — though Wan2.5-I2I-Preview still achieves a higher score of 63.4 in that role.

Future Directions

1. Closing the gap to proprietary models: Wan2.5-I2I-Preview reaches 63.4 overall as a refiner versus 54.9 for the trained open-source model, so stronger open refinement training remains an open problem.
2. Improving aesthetic quality: Even ground-truth reference images score only 4.2 on the 0–10 Aesthetic Quality dimension, and the best pipeline result in that dimension is 4.9, suggesting aesthetic scoring and generation are both under-explored.
3. Generalizing the self-correcting loop: The authors confirm the pipeline structure is sound and the bottleneck is model capability, so applying RL-tuned refinement to other structured generation tasks (text rendering, posters, infographics) is a natural extension.
4. Benchmark depth: The paper states that detailed benchmark statistics and dimension descriptions are provided in the Appendix, which is truncated here; expanding TableVisBench beyond 800 instances and refining error-quantification scoring are logical next steps.

Target Audience

Researchers and engineers working on multimodal generation, unified image-generation models, reinforcement learning for diffusion models, and automated visualization. It is also relevant to practitioners in graphics design automation, slide/poster generation, and scientific figure creation who need both data fidelity and visual quality, and to anyone building reflection- or agent-style self-correcting generation pipelines.

Authors’ abstract

While existing generation and unified models excel at general image generation, they struggle with tasks requiring deep reasoning, planning, and precise data-to-visual mapping abilities beyond general scenarios. To push beyond the existing limitations, we introduce a new and challenging task: creative table visualization, requiring the model to generate an infographic that faithfully and aesthetically visualizes the data from a given table. To address this challenge, we propose ShowTable, a pipeline that synergizes MLLMs with diffusion models via a progressive self-correcting process. The MLLM acts as the central orchestrator for reasoning the visual plan and judging visual errors to provide refined instructions, the diffusion execute the commands from MLLM, achieving high-fidelity results. To support this task and our pipeline, we introduce three automated data construction pipelines for training different modules. Furthermore, we introduce TableVisBench, a new benchmark with 800 challenging instances across 5 evaluation dimensions, to assess performance on this task. Experiments demonstrate that our pipeline, instantiated with different models, significantly outperforms baselines, highlighting its effective multi-modal reasoning, generation, and error correction capabilities.