Decompose Radicals, Then Reward: Fine-Grained Inspection for Accurate Chinese Text Rendering

AI summary

Overview

• Research area: Computer vision, specifically text-to-image generation and reward design for reinforcement-learning post-training of image generators, focused on rendering Chinese visual text.
• Technical level: Advanced. The paper assumes familiarity with diffusion/flow-based image generation, GRPO-style group-relative policy optimization, OCR pipelines, and Chinese character decomposition.
• Scope: The paper proposes IDSpect, a target-conditioned reward that decomposes Chinese target text into Ideographic Description Sequence (IDS) tokens and rewards partial structural matches at the component and spatial-relation level, evaluated through GRPO post-training of Qwen-Image on LongText and GenTextEval.

What This Paper Is About

Text-to-image models still struggle to render accurate Chinese characters. Existing reinforcement-learning rewards use OCR to turn a generated image into a transcript and then compare that transcript to the target string, which treats each Chinese ideograph as one atomic symbol. Because a Chinese character is actually built from reusable components arranged under explicit spatial relations (left–right, top–bottom, enclosing, and others), this atomic comparison gives no credit when a model fixes one radical but still gets the whole character wrong. The authors build a recognizer that transcribes rendered text directly into IDS and use it as the structural half of a new reward.

Key Contributions

1. Identifying the coarse-credit limitation of atomic OCR rewards. The authors show that structurally distinct intra-character errors can receive indistinguishable feedback under character-level rewards, citing the example in Figure 1 where a specialist OCR model assigns 0.00 to a plausible-but-wrong character, an MLLM OCR assigns a false-positive 1.00 by recovering the target from context, and TextPecker assigns 0.00 after collapsing the glyph into an anomaly marker.
2. An expert IDS recognizer. They train a model that maps a rendered text-region image directly to a sequence of spatial operators and character components, coupling an SVTRv2 encoder with a Transformer-style decoder following NRTR, rather than recognizing characters first and decomposing them symbolically afterward.
3. IDSpect, a fine-grained reward for GRPO. IDSpect deterministically decomposes the target text into IDS tokens, aligns each crop-level visual IDS prediction to a span of the target sequence with globally unique token credit, and combines this structural term with a whole-character semantic reward.
4. A balanced synthetic training corpus, IDSynth-1M. To prevent the recognizer from taking character-frequency and language-modeling shortcuts, they adapt the UnionST rendering engine to build one million rendered scene text images with roughly uniform character sampling and paired IDS targets.

Main Findings

• IDSpect leads on structural quality and semantic alignment. On GenTextEval, IDSpect reaches 0.979 Qua. and 0.911 Sem., exceeding the OCR reward by 0.026 and 0.037 respectively and improving over the TextPecker reward by 0.006 and 0.014.
• On LongText, IDSpect achieves the best structural and semantic numbers. It reaches 0.975 Qua. and 0.928 Sem., while its Avg. score of 0.972 is described as comparable to TextPecker's 0.974.
• GRPO post-training lifts the frozen model substantially. On LongText, OCR-guided GRPO raises the original benchmark text score from 0.920 to 0.967, with Qua. and Sem. moving from 0.924 and 0.834 to 0.956 and 0.886. TextPecker-guided GRPO reaches 0.974 Avg. on LongText with 0.969 Qua. and 0.908 Sem., and 0.973 Qua. with 0.897 Sem. on GenTextEval.
• Partial structural credit is the distinctive mechanism. In the Figure 1 case, IDSpect retains credit for matched IDS tokens while penalizing mismatched ones, assigning a standalone IDS reward of 0.22 where character-level rewards produced 0.00 or a false-positive 1.00.
• Target-grounded structural supervision beats OCR-derived consistency on semantics. In the GenTextEval ablation, per-crop consistency against the OCR prediction gives the highest Qua. of 0.980 but only 0.901 Sem., below the 0.911 achieved by target-based crop-wise alignment.
• Alignment strategy matters as much as the representation. Crop-wise alignment reaches the highest Sem. of 0.911, improving over direct concatenation and character-wise matching by 0.025 and 0.012 respectively, with a Qua. of 0.979 that is only 0.001 below per-crop consistency.
• Qualitative gains concentrate in long text. In the three-prompt comparison of Figure 3, gains are described as especially apparent in longer lines on signs and posters, where structural errors accumulate across characters.
• No inference-time overhead. Both reward branches are discarded after training, so IDSpect does not change the image generator or add cost at inference.

Methodology in Plain English

The authors keep the image generator untouched and redesign only the reward used during post-training. First, they build an IDS representation: a Chinese character is recursively decomposed into spatial operators and reusable components, using the Unicode 16.0 BabelStone lexicon over characters covered by GB 18030–2022, with recursion capped at a depth of 10 and the decoder vocabulary drawn from the resulting operators and components. Because natural scene-text corpora are skewed toward frequent characters, they render a balanced synthetic set of one million images (IDSynth-1M) with roughly uniform character sampling, which also suppresses natural lexical co-occurrence so the recognizer must read structure from pixels instead of guessing from context. The maximum decoded IDS sequence length is 100 tokens.

At training time, an OCR detector extracts text crops from each generated image. One branch compares the OCR transcript with the target text using a normalized character edit similarity that gives full credit when the target occurs as a substring and otherwise uses a truncated edit distance, keeping the score in [0,1]. The other branch feeds each crop to the IDS recognizer, then aligns the predicted token sequence to candidate spans of the target's IDS sequence via semi-global edit alignment, filters near-duplicates with non-maximum suppression, and selects a global assignment in which each target token can be credited at most once. The structural reward is a token-level F1: twice the number of uniquely credited target tokens divided by the sum of the target IDS length and the total predicted length. The two rewards are combined with weight λ = 0.5.

Post-training uses Flow-GRPO through the Flow-Factory framework, optimizing LoRA adapters of rank 64 with scaling factor 128 under AdamW, learning rate 3×10⁻⁴ and weight decay 10⁻⁴, with all other hyperparameters at framework defaults. Baselines are the frozen Qwen-Image model, OCR-only GRPO, and TextPecker-guided GRPO; results are reported on LongText-Bench and GenTextEval-Bench using the original benchmark text score (Avg.), structural quality (Qua.), and semantic alignment (Sem.), with Qua. and Sem. evaluated by TextPecker.

Why This Matters

Impact on research. The paper argues that for Chinese text rendering the bottleneck is not only the reward signal's accuracy but its granularity. It reframes reward design as a compositional matching problem over components and spatial operators, and it shows that how predicted structures are aligned with the target (crop-wise alignment versus concatenation or character-wise matching) changes semantic results measurably. It also connects Chinese OCR work on radicals, strokes, and IDS, previously used for recognizing rare or unseen characters, to the very different setting of rewarding a generative model.

Real-world applications.

• Generating posters, signs, and advertisements that contain long Chinese headlines, where errors compound across characters.
• E-commerce and product-image creation, where brand names and specification text must be rendered exactly.
• Localization and marketing pipelines that produce Chinese-language creative assets at scale.
• Assistive and educational material production where incorrect characters are not merely cosmetic.

Industry relevance. Because both reward branches are dropped after training, IDSpect can be added to an existing post-training pipeline without modifying the deployed generator or paying any inference-time cost. That combination of a training-only change and measurable gains on two public benchmarks is what makes it practical for teams already running GRPO-style post-training on their own image models.

Future Directions

• Extending the decomposition framework to broader Chinese character coverage beyond what was tested.
• Applying the same IDS-style structural supervision to multilingual scripts, specifically Japanese and Korean, as the authors state in their conclusion.
• Investigating the trade-off exposed by the ablation: per-crop consistency achieved the highest structural quality (0.980) but weaker semantics (0.901), while target-based crop-wise alignment reached 0.911 Sem. and 0.979 Qua., leaving open how best to combine both signals.
• Determining whether the balanced synthetic data recipe behind IDSynth-1M generalizes to other long-tailed writing systems or to recognizer backbones other than SVTRv2 with an NRTR-style decoder.

Target Audience

Researchers and engineers working on diffusion or flow-based text-to-image generation, reinforcement-learning post-training and reward modeling, and scene text recognition for Chinese or other compositionally structured scripts. It is also relevant to practitioners who need reliable Chinese text in generated marketing or product imagery, and to readers interested in how OCR-style evaluation can be turned into a differentiable-enough training signal. Familiarity with GRPO and OCR pipelines will help, since the paper moves quickly between reward formulation and recognition architecture.

Authors’ abstract

Rendering accurate Chinese text remains challenging for text-to-image models. Existing OCR-based reinforcement-learning rewards compare decoded transcripts with target strings. Such rewards overlook the compositional nature of Chinese writing: an ideograph consists of reusable components arranged through explicit spatial relations, yet OCR evaluates it as an atomic character. Consequently, visually different radical-level errors may receive equally coarse feedback, encouraging glyphs that merely resemble the target instead of faithfully reproducing its internal structure. We employ Ideographic Description Sequences (IDS), which comprise spatial operators and character components, and train an expert IDS recognizer to transcribe rendered Chinese text into this representation. Building on this recognizer, we introduce IDSpect, which deterministically decomposes the target text into IDS tokens and aligns crop-level visual IDS predictions with the target sequence. Globally unique token credit makes this comparison robust to the order of detected text regions. Combined with a whole-character semantic reward, IDSpect supplies fine-grained credit with component and spatial-relation without changing the image generator or adding inference-time cost. Experiments with GRPO post-training of Qwen-Image demonstrate that IDSpect achieves leading structural quality and semantic alignment on LongText and GenTextEval.