TextVerifier: Seeing Beyond Recognition to Verify Chinese Text Rendering

AI summary

Overview

Research area: Computer vision and generative AI — specifically the evaluation and post-training of text-to-image models that render Chinese text, sitting at the intersection of Chinese character structure representation, conditional anomaly detection, and reinforcement-learning-based image generation.

Technical level: Advanced. The paper assumes familiarity with CLIP-style contrastive encoders, InfoNCE and hinge losses, diffusion/flow-matching post-training (Flow-GRPO), and Chinese character decomposition (IDS).

Scope: The paper builds a target-conditioned verifier that decides whether an image crop is a correct rendering of a specified Chinese character, evaluates it on a new per-character benchmark, and uses its score as a reward signal.

What This Paper Is About

Text-to-image models now generate readable Chinese sentences, but individual characters can contain local defects — an extra or missing stroke, or a malformed component — and a sentence still reads correctly at a glance. OCR and vision-language models are trained for recognition and are therefore deliberately invariant to the very visual deviations that define a rendering error, so they do not catch these mistakes. The paper formalizes "text rendering verification" as a target-conditioned problem: given a target character and its image crop, decide whether the crop is a correct rendering of that character, and then use that decision to both measure and reduce rendering errors.

Key Contributions

1. TextVerifier: A structure-aware verifier that encodes each character's Ideographic Description Sequence (IDS) decomposition inside a CLIP-style dual tower, reaching high accuracy at only 0.36B parameters.
2. A three-stage training curriculum with label-free supervision: combining synthetic positive–negative pairs (font synthesis and Makemeahanzi-based stroke edits) with hard "same-character, different-quality" pairs mined from the training trajectory of a text-region inpainting model.
3. TextRenderBench-ZH: The first per-character benchmark for generated Chinese text verification, labeling position, identity, and correctness for 7,358 characters drawn from 308 generated images, of which 642 are confirmed rendering errors.
4. A reward application: demonstrating that the verifier's per-character score can serve as a reward for Flow-GRPO post-training, reducing rendering errors while preserving diversity and aesthetics better than an 8B-VLM reward.

Main Findings

• Verifier accuracy: TextVerifier reaches an F1 of 0.844 after all three stages (precision 0.830, recall 0.858), versus OCR confidence at 0.526 (oracle best threshold), a general VLM at 0.329, and TextPecker at 0.590.
• OCR is confidently wrong: PP-OCRv6 assigns confidence above 0.99 even to a clearly wrong character, and its confidence peaks at only 0.53 F1 even at an oracle threshold. At threshold 0.90 it scores 0.614 precision, 0.347 recall, 0.444 F1.
• General VLMs fail in both directions: Qwen3.8-Max (F1 0.198) and GPT-5.6-sol (F1 0.101) almost never fire, while Gemini 3.7 Flash fires often but wrongly (precision 0.236, recall 0.540, F1 0.329); Gemini 3.7 Flash recalls only 54% of erroneous characters.
• TextPecker produces many false alarms: TextPecker (Qwen3-VL-8B) reaches recall 0.765 at precision only 0.480 (F1 0.590); the InternVL3-8B variant reaches F1 0.433.
• Size and speed: TextVerifier's scoring model is 0.36B parameters, 23 times smaller than TextPecker, and runs at 7.6 ms per character, 2.3 times faster than PP-OCRv6 recognition at 17.2 ms, and orders of magnitude faster than the TextPecker VLMs (320.4 ms and 276.0 ms).
• IDS matters more than a character token: Replacing IDS input with a single-character token drops F1 by about 10.4, 7.1, and 10.1 points across the three stages, with Stage-3 recall falling from 0.858 to 0.648 while precision even rises slightly.
• Every training component contributes: Removing the Stage-1 InfoNCE replay costs 4.8 points (0.844 to 0.796), removing the Stage-2 threshold loss costs 2.7 points (to 0.817), skipping Stage 2 entirely costs 3.3 points (to 0.811), and with no Stage-1 pretraining the model collapses to 0.111.
• As a reward, glyph awareness is essential: On oneID-ZH, a recognition-only reward (NED-only, dropping the glyph term) barely lifts Glyph-Acc over Base (0.812 to 0.822), whereas the full reward reaches 0.915.
• Competitive with a much larger reward model: TextVerifier's reward attains comparable glyph accuracy to TextPecker (0.915 vs. 0.919) while perturbing diversity and aesthetics less (DreamSim 0.1753 vs. 0.1651; ImageReward 0.6330 vs. 0.6288), with a 23 times smaller reward model (0.36B vs. 8.29B).

Methodology in Plain English

Reframing the task. Instead of asking a model "what does this say," the authors ask "is this a legitimate way to write this known character?" The target character is supplied by an upstream recognizer and treated as a condition, not something to predict. The framing is conditional anomaly detection: each character has a manifold of legitimate renderings across fonts, calligraphy, and styles, and an error is a small departure from it.

Architecture. The model is a CLIP-style two-tower network. The image tower is a ViT-L/16 whose patch tokens are averaged; the text tower is a DeBERTa-v2 large trained from scratch, and it reads the character's Ideographic Description Sequence rather than a single-character symbol, so characters sharing components or structure sit close in sequence space. Both towers project into a shared, ℓ2-normalized 1024-dimensional space, and the match is the cosine similarity.

Three stages of data and training. Stage 1 uses real data: CRAFT detects characters, PP-OCRv6 recognizes them, and this yields millions of crop–IDS pairs for a bidirectional InfoNCE that teaches identity, with a "same-character masking" tweak so other crops of the same character are not treated as negatives. Stage 2 adds synthetic hard negatives with absolute labels — correct renderings are anchored above a threshold and synthesized errors below it, each with a margin. Stage 3 uses the insight that a text-region inpainting model still training leaves "same-character, different-quality" pairs along its trajectory: an early checkpoint renders strokes wrongly, a late checkpoint renders them mostly correctly, with only the glyph changing. Because these labels are only relative, Stage 3 uses a truncated-hinge loss that stops updating once pairs are correctly ordered.

Scoring at inference. Rather than thresholding a single similarity, the method keeps K image-space prototypes per character (k-means over 500 normal crops, K = 10) and scores a crop by its cosine distance to the nearest prototype; the threshold is the 90th percentile of the training distribution.

As a reward. The score is mapped to a [0,1] reward that is 1 for a clearly correct glyph, 0 for a clearly wrong one, and linear in between. Per image, the glyph term combines a mean with a worst-case minimum (0.9/0.1), a recognition term based on normalized edit distance is added, and the combined reward (0.8 glyph + 0.2 recognition) is fed to Flow-GRPO.

Why This Matters

Impact on research. The paper separates recognition from verification and argues they pursue opposite objectives — invariance versus sensitivity to structural deviation. It provides a per-character benchmark and annotation protocol where previously evaluation rested on OCR string metrics over prompt-only benchmarks, and it shows that a small, fast, continuous, calibrated score can substitute for an expensive VLM judge in reinforcement-learning post-training.

Real-world applications:

• Publishing and advertising, where a single malformed character in a headline can make an entire generated image unusable.
• Education and Chinese language learning materials, where character correctness is essential and stroke-level errors matter.
• Automated quality control for text-to-image generation pipelines, filtering or flagging outputs before human review.
• Post-training and fine-tuning of generative models, where the verifier supplies a cheap per-character reward.

Industry relevance. The verifier is 0.36B parameters, runs at 7.6 ms per character, and is 2.3 times faster than PP-OCRv6 recognition, making per-character verification cheap enough to run at scale or inside a training loop — something the 8B TextPecker backbone (320.4 ms per character in the Qwen3-VL-8B variant) is not. Matching an 8B reward on error reduction while perturbing diversity and aesthetics less, and at 23 times fewer parameters, is directly relevant to cost-sensitive deployment of text-to-image systems.

Future Directions

• Whether the verifier's sensitivity transfers to other scripts with dense, component-based structure, or is specific to Chinese IDS decomposition.
• How the approach scales beyond the 6,500 level-1/2 characters emphasized in the data, and how the heavily long-tailed character distribution (most frequent character ≈110K crops, least frequent 1) affects rare characters.
• Whether the label-free inpainting-trajectory trick generalizes to other generators, or depends on the specific inpainting model trained here.
• Whether the verifier can be extended from per-character correctness to line- or paragraph-level layout and interaction errors, which the paper's error criterion does not appear to cover.

Target Audience

Researchers and engineers working on text-to-image generation, especially Chinese text rendering; teams building evaluation or reward models for generative image systems; practitioners who need automated, character-level quality control for generated text in images; and readers interested in conditional anomaly detection or in using structured linguistic representations (IDS) inside vision-language encoders. Readers without background in contrastive representation learning or reinforcement-learning post-training will find the technical sections demanding but the framing accessible.

Authors’ abstract

Text-to-image models increasingly generate readable Chinese text that still contains local rendering defects, such as a character with an extra or missing stroke or a malformed component. Such errors are invisible to OCR and vision-language models, which are trained for recognition and are therefore invariant to the very structural deviations that define a rendering error. We formalize text rendering verification as a target-conditioned problem: given a target character and an image crop, decide whether the crop is a correct rendering of that character. Our verifier TEXTVERIFIER feeds the text tower a character's Ideographic Description Sequence (IDS) rather than a single-character symbol, and is trained on 12.7M crop-IDS pairs plus label-free "same-character, different-quality" hard pairs sampled from the training trajectory of a text-region inpainting model. To evaluate this ability we build TEXTRENDERBENCH-ZH, the first per-character benchmark for generated Chinese text, annotating position, identity, and correctness for 7,358 characters. TEXTVERIFIER reaches an F1 of 0.844, far above OCR confidence (0.526), a general VLM (0.329), and TextPecker (0.590). Used as a per-character reward for Flow-GRPO post-training, it further reduces rendering errors, showing that its score can both detect and correct errors.