UniRec-0.1B: Unified Text and Formula Recognition with 0.1B Parameters

AI summary

Overview

• Research area: Computer vision / document AI — optical character recognition (OCR) and mathematical formula recognition, with an emphasis on unified vision-language modeling for document parsing.
• Technical level: Intermediate. The paper is readable without deep math, but familiarity with encoder-decoder models, tokenizers, and document parsing pipelines helps.
• Scope: The paper proposes a 0.1B-parameter model, a 40-million-sample training set, and a new benchmark for jointly recognizing text and formulas across multiple structural levels, languages, and document domains.

What This Paper Is About

Document parsing systems must read both ordinary text and mathematical formulas, but current vision-language models that unify these tasks rely on billions of parameters, making them slow and expensive. The authors observe that text and formulas make up the overwhelming majority of document content yet saturate in accuracy at a very small model size, so they build a tiny specialist model — UniRec-0.1B — dedicated to those two modalities, rather than one giant model that handles everything. To train it, they assemble a large-scale bilingual dataset and design two techniques that help a small model handle multi-level structure and the semantic overlap between prose and formulas.

Key Contributions

1. UniRec40M dataset. A large-scale Chinese and English dataset containing approximately 40 million text, formula, and mixed text-formula samples, built from arXiv TeX sources, Wikipedia pages, digital-born PDFs, and public scene-text, handwritten, and examination datasets. It is described as filling a gap in multi-level, bilingual, unified text-formula recognition data.

2. UniRec-0.1B model. A 0.1B-parameter encoder-decoder recognition model that performs text and formula recognition at character, word, line, paragraph, and multi-paragraph levels, trained from scratch without loading pre-trained weights.

3. Two training/modeling techniques. Hierarchical Supervision Training (HST), which inserts line-level and paragraph-level supervision tokens so the model learns structural dependencies, and a Semantic-Decoupled Tokenizer (SDT), which builds separate text and formula vocabularies to prevent cross-modality token confusion. The paper notes that existing systems such as PaddleOCR-VL, MinerU2.5, MonkeyOCR, and Dolphin-1.5 lack such designs.

4. UniRec-Bench and extensive evaluation. A benchmark built by extending OmniDocBench with extracted text, formula, and mixed text-formula blocks, categorized into five hierarchical levels, two languages, and nine document domains, used to compare UniRec-0.1B against expert recognizers, document parsing models, and general VLMs.

Main Findings

• Text and formulas dominate document content. Text and formulas account for 97.43% of page regions in quantity in OmniDocBench, and consume 36,839 seconds in total, or 87.90% of total parsing time when using MinerU2.5.

• Accuracy saturates near 0.1B parameters for text and formulas. The scaling-law analysis shows text and formula recognition improving rapidly with model size and peaking around 0.1B parameters, while table performance keeps improving with larger models. This motivates a divide-and-conquer strategy instead of one large unified model.

• Strongest average accuracy on UniRec-Bench. UniRec-0.1B achieves an average edit distance of 0.071 with 0.1B parameters, compared with 0.087 for PaddleOCR-VL (0.9B), 0.091 for DeepSeek-OCR (3B-A0.5B), 0.107 for dots.ocr (3B), 0.110 for Dolphin-1.5 (0.3B), 0.138 for MinerU2.5 (1.2B), and 0.109 for the pipeline-based PP-OCRv5 (42M).

• Large gains over formula recognition experts. UniRec-0.1B reaches 0.038 on formula recognition, outperforming Mathpix (0.322), Pix2Tex (0.337), and UniMERNet-B (0.238) by 18.8%, 20.3%, and 10.4% respectively.

• Clear advantage over a comparable-size parsing model. Against Dolphin-1.5 (0.3B), UniRec-0.1B improves by 1.2% on text, 23.1% on formulas, and 7.4% on mixed content, and is more than 2× faster.

• Both techniques contribute measurably. Removing HST worsens text by 1.2%, formulas by 1.0%, and mixed content by 1.5%; the five text levels degrade by 0.5%, 0.8%, 0.4%, 1.7%, and 1.2%, with the largest losses at paragraph and multi-paragraph levels. Removing SDT costs 1.2% on text, 11.1% on formulas, and 1.8% on mixed content. Combined, the two techniques produce an average gain of 2.4%, with the largest domain gains on PPT2PDF (2.8%) and Exam Paper (2.9%).

• Speedups of 2-9× across settings. Block-level inference drops from 1.88 s for PaddleOCR-VL to 0.37 s for UniRec-0.1B, and page-level from 31.92 s to 6.20 s — both over 5× faster. Dolphin-1.5 takes 0.78 s per block and 13.16 s per page; MinerU2.5 takes 2.54 s and 42.72 s; DeepSeek-OCR takes 3.47 s and 58.95 s; MonkeyOCR takes 3.62 s and 58.39 s.

• Improves full-page parsing when swapped into existing pipelines. Replacing the text and formula recognition modules of MinerU2.5 with UniRec-0.1B improves full-page edit distance from 0.143 to 0.120 (a 2.3% gain) and cuts page parsing time from 42.72 s to 6.2 s, nearly a 7× speedup. With PaddleOCR-VL, the full-page edit distance improves by a further 0.2%, reaching a new state of the art. A collaborative setup using UniRec-0.1B for text and formulas plus MinerU2.5-1.2B for tables yields roughly 4× overall speedup versus MinerU2.5-1.2B alone.

• Domain-specific strengths. UniRec-0.1B achieves the best accuracy in the Note domain, surpassing PaddleOCR-VL by 1.2%, and also in the Newspaper domain, suggesting better robustness to handwriting and to blurred or degraded text than most competing methods.

• Fine-grained recognition remains a weak point for page-level end-to-end models. The benchmark shows end-to-end full-page models such as dots.ocr, Nanonets-OCR2, and DeepSeek-OCR performing worse at character, word, and line levels, while multi-stage methods do better at those fine-grained levels.

Methodology in Plain English

The authors first build training data at scale. They download arXiv TeX sources and Wikipedia pages converted to TeX, insert LaTeX color commands so every text or formula token gets a unique color, render the files to PDFs, and then align colors between source and rendering to recover labels at word and line levels, with paragraph labels recovered by parsing LaTeX grammar. They add digital-born PDFs (industry research reports, newspapers) processed with PyMuPDF, plus public datasets for Chinese scene text, English scene text, handwriting, and exam-style content, along with handwritten notes labeled by a large vision-language model (Qwen3VL-235B-A30B) and manually refined. Training epochs use a proportion-balanced sampling strategy that keeps stable ratios among text, formula, and mixed samples.

The model itself is a standard encoder-decoder. A FocalNet image encoder processes the image at native resolution with width capped at 960 and height at 1408 pixels, producing a feature map downsampled by 32× with dimension D = 768, which is flattened into visual tokens. A six-layer Transformer decoder with cross-attention (hidden size D, D/64 attention heads) generates the output sequence autoregressively under a causal mask, trained with cross-entropy loss.

Two design choices make the small model work. Hierarchical Supervision Training inserts special tokens into the label sequence — one marking a line break within a paragraph and one marking the end of a paragraph — so the model explicitly learns layout hierarchy; at inference these are stripped out and the paragraph marker is converted into newline characters. The Semantic-Decoupled Tokenizer trains two separate tokenizers, one on plain text and one on formulas, then adds the formula vocabulary into the text tokenizer as special tokens (excluding duplicates). This keeps tokens that look identical in both modalities, such as those for summation, infinity, or fraction commands, from sharing a single embedding.

Evaluation uses UniRec-Bench, created by pulling text, formula, and mixed text-formula blocks out of OmniDocBench full pages and sorting them into five levels, two languages, and nine domains, plus full-page evaluation on OmniDocBench itself. Accuracy is measured as edit distance between prediction and ground truth. Training used AdamW with weight decay 0.01, a learning rate of 1×10⁻⁴, global batch size 64, a one-cycle schedule with 0.5-epoch linear warm-up across 10 epochs, maximum token length 1024, and a vocabulary of 56,371 tokens, taking about 80 hours on 8 A800 40GB GPUs.

Why This Matters

The paper argues that unifying every document task in one giant model is wasteful, because the dominant modalities (text and formulas) gain almost nothing from extra parameters while still dominating inference time. Showing that a 0.1B model can match or beat billion-scale systems reframes the efficiency-accuracy trade-off for document AI.

Real-world applications:

• Document understanding and information retrieval: faster, cheaper conversion of scanned and digital documents into machine-readable text and formulas.
• Digital education: accurate extraction of exam papers, textbooks, and K-12 materials, including handwritten notes — the paper reports its largest domain gains on Exam Paper and PPT2PDF.
• Scientific and technical publishing: turning arXiv papers and technical reports into structured text and LaTeX formulas at scale.
• Archival and records processing: handling newspapers and degraded scans, where the paper reports UniRec-0.1B achieving the best accuracy.

Industry relevance: the model is designed as a drop-in replacement for the recognition stage of existing two-stage parsing systems. Substituting it into MinerU2.5 and PaddleOCR-VL improved both accuracy and speed without changing layout analysis or table recognition, which is a practical path to cheaper large-scale document processing deployments.

Future Directions

• Improving fine-grained recognition. The paper states that benchmark results reveal limitations in existing models, such as excelling at paragraph-level recognition while struggling with fine-grained recognition, and names these as future research directions.

• Extending the divide-and-conquer idea to other modalities. Table recognition continues to benefit from larger models, which raises the question of how far lightweight specialist models can be pushed before a larger model is genuinely needed, and how the specialists should be coordinated.

• Robustness to handwriting and visually degraded inputs. UniRec-0.1B performs best on Note and Newspaper domains while other models lag significantly, indicating that most current methods are biased toward digital-born documents and that noisy or degraded inputs remain an open problem.

• Broader dissemination and reuse. The authors release the codebase and dataset at github.com/Topdu/OpenOCR and express the hope that UniRec-0.1B, UniRec40M, and UniRec-Bench will advance the document parsing field — making external validation of the dataset, the benchmark, and the tokenizer design an obvious next step.

Target Audience

Researchers and engineers working on OCR, document parsing, and document understanding; practitioners building or deploying document-processing pipelines who care about inference cost; and developers interested in how far small, task-specialized vision-language models can go compared with billion-parameter general models. Readers focused on table or chart recognition will find the paper less directly relevant, since it deliberately excludes those tasks.

Authors’ abstract

Text and formulas constitute the core informational components of many documents. Accurately and efficiently recognizing both is crucial for developing robust and generalizable document parsing systems. Recently, vision-language models (VLMs) have achieved impressive unified recognition of text and formulas. However, they are large-sized and computationally demanding, restricting their usage in many applications. In this paper, we propose UniRec-0.1B, a unified recognition model with only 0.1B parameters. It is capable of performing text and formula recognition at multiple levels, including characters, words, lines, paragraphs, and documents. To implement this task, we first establish UniRec40M, a large-scale dataset comprises 40 million text, formula and mixed samples, enabling the training of a powerful yet lightweight model. Secondly, we identify two challenges when building such a lightweight but unified expert model. They are: structural variability across levels and semantic entanglement between textual and formulaic content. To tackle these, we introduce a hierarchical supervision training that explicitly guides structural comprehension, and a semantic-decoupled tokenizer that separates text and formula representations. Finally, we develop a comprehensive evaluation benchmark covering Chinese and English documents from multiple domains and with multiple levels. Experimental results on this and public benchmarks demonstrate that UniRec-0.1B outperforms both general-purpose VLMs and leading document parsing expert models, while achieving 2-9x speedup, validating its effectiveness and efficiency. Codebase and Dataset: https://github.com/Topdu/OpenOCR.