FocusVTC: Efficient and High-Performance Visual Text Compression with Adaptive Resolution

AI summary

Overview

• Research area: Computer vision and multimodal long-context efficiency — specifically visual text compression (VTC), where text is rendered as page images so a vision-language model can read it with fewer input tokens.
• Technical level: Advanced. The paper assumes familiarity with vision-language models, supervised fine-tuning, reinforcement learning for LLMs (GRPO), KV-cache and attention costs, and document-grounded reasoning benchmarks.
• Scope: The paper introduces FocusVTC, an adaptive-resolution VTC framework that keeps a compressed low-DPI global view of a document and selectively enhances only question-relevant regions at high DPI during reasoning, trained with multi-resolution supervised fine-tuning and GRPO over 29.4K Reasoning–Evidence Localization chain-of-thought examples.

What This Paper Is About

Long-context reasoning in LLMs is expensive in attention computation, latency, and KV-cache memory. Visual text compression reduces the input length by rendering text as images, but fixed-resolution rendering forces a trade-off: low DPI saves tokens yet makes characters hard to read, while high DPI makes text legible but spends tokens across the whole page, including irrelevant passages. FocusVTC's goal is to break that trade-off by rendering the document once at low DPI and then spending extra tokens only on the small regions the question actually depends on.

Key Contributions

1. An adaptive-resolution framework for VTC. It combines a compressed low-DPI global context with tool-mediated access to high-resolution content for selected document regions, decoupling whole-page token cost from local legibility.
2. The REL-CoT dataset. 29.4K high-quality Reasoning–Evidence Localization chain-of-thought examples, each linking a reasoning trace and answer to page indices and normalized bounding boxes, verified through a three-stage cross-model pipeline.
3. A two-stage training recipe without continual pretraining. Multi-resolution REL supervised fine-tuning (REL-SFT) across seven rendering resolutions teaches evidence localization, and Group Relative Policy Optimization (GRPO) learns when to enhance resolution, how to use the resulting observations, and when to stop.
4. Empirical results with preserved general capabilities. 87.4 on RULER v1 at 2.9 times compression, 56.40 on LongBench, a 13.91-point MRCR macro-average gain, 51.19 on VTCBench, with MMMU rising from 65.12 to 66.73 and MME from 2424.02 to 2457.62.

Main Findings

• RULER v1 at low DPI: At 72 DPI, FocusVTC scores 87.4 at 2.9 times input compression (including tool observations), versus 57.5 for Glyph at 3.0 times compression. On RULER v1/v2 at 72 DPI it reaches 87.38/75.94, compared with 37.40/44.64 for the same backbone without crops.
• LongBench matches and exceeds text input: FocusVTC scores 56.40 overall versus 55.86 for its text-input backbone Qwen3.5-9B, and 52.34 for the strongest VTC baseline, Glyph. It gains 20.54 points over the same backbone at a fixed 72 DPI. Gains concentrate on QA and synthetic retrieval; summarization changes little, and few-shot results are mixed.
• MRCR: FocusVTC's two/four/eight-needle averages are 60.76/45.21/30.71 versus 42.63/25.97/16.57 for Glyph, and its macro-average over six length bins and needle settings improves from 31.65 to 45.56 (+13.91). In the longest bin, the text reference is 197,909 tokens and the prompt is 53,181 tokens; adding two/four/eight-needle observations of 7,296/10,290/13,048 tokens gives 3.3/3.1/3.0 times compression.
• Latency: On 64K–128K four-needle MRCR examples, FocusVTC achieves a 2.79 times online end-to-end speedup over Qwen3.5-9B Text, excluding offline page rendering.
• VTCBench: FocusVTC achieves a 51.19 macro-average across Retrieval, Reasoning, and Memory. It obtains the best Retrieval (91.00) and Memory (26.33) averages; Reasoning improves over the fixed-resolution backbone in the two longest bins even though its overall average (36.25) remains lower.
• General multimodal capabilities: MMMU increases from 65.12 to 66.73 and MME from 2424.02 to 2457.62.
• Training ablations: REL-SFT is a useful initialization for GRPO — FocusVTC improves eight of nine reported aggregates over FocusVTC w/o SFT, including LongBench (49.30 to 56.40), with VTCBench Reasoning the sole exception (38.15 to 36.25). Tool access without training hurts: Qwen3.5-9B+tools and FocusVTC w/o GRPO+tools score lower than their no-tool counterparts; for FocusVTC w/o GRPO, enabling tools reduces LongBench from 37.86 to 25.41. After GRPO, FocusVTC surpasses FocusVTC w/o GRPO+tools on all nine aggregates, including a rise from 22.98 to 87.38 on RULER v1, and beats FocusVTC w/o tools on LongBench (56.40 versus 36.82).
• Font and resolution choices matter: DejaVu Sans at 9 pt with 1 pt additional line spacing is selected from 15 fonts and eight point sizes; it costs 8,368.4 visual tokens per 32K-token context versus 8,417.8 for Verdana, has close overall CER (3.91% versus 3.85%) but 0.30 percentage points lower CER on confusable sequences (95% CI [-0.52, -0.09]), and improves downstream LongBench from 54.93 to 56.40. At 72 DPI on RULER v1, FocusVTC uses an average of 2,154 prompt tokens and 723 extra-observation tokens (about 2,877 total) versus an 8,400-token text context; starting at 144 DPI costs 2.8/2.6 times as many input tokens for only modest gains.
• GRPO progression: Across four stages, the policy first increases tool calls and response length as grounding IoU rises, then reduces calls while responses stay long and IoU briefly dips, and finally shortens responses while IoU rises and benchmark scores reach their strongest checkpoint.

Methodology in Plain English

The researchers render a text document into page images at a controlled font, point size, and DPI. The model starts with a compressed, low-DPI view of every page. As it reasons, it can call a tool that re-reads a selected page and bounding box from an aligned 144-DPI version of the same page, and it continues reasoning with that sharper crop in its history until it answers or hits a call budget.

To teach this behavior, they built training data in three stages with different models: one filters out questions answerable from general knowledge so that the model cannot shortcut reading, one annotates reasoning traces with page indices and bounding boxes, and a third independently verifies that the selected high-resolution regions actually support the answer. The result is roughly 29.4K verified examples.

Training has two phases. First, supervised fine-tuning on the same examples rendered at seven different DPIs (48, 60, 72, 84, 96, 120, 144) teaches the model to localize evidence regardless of the starting resolution. Second, GRPO optimizes whole interaction trajectories using a reward that is 0.8 times answer correctness plus 0.2 times output validity, plus a tool-use bonus that only applies when the answer is fully correct. That bonus rewards enhancing when the initial view is compressed, localizing regions precisely (measured by IoU against annotated boxes, with a penalty for overly broad regions), and avoiding redundant calls.

Why This Matters

• Impact on research: The paper argues that global context coverage and precise local reading do not need the same resolution. Turning reading detail into an explicit, learned, tool-mediated decision — rather than a fixed rendering setting — reframes the compression–performance trade-off in visual text compression and connects it to broader work on efficient long-context inference and grounded visual reasoning.
• Real-world applications:
  • Long-document question answering over contracts, filings, or manuals where only a few passages matter.
  • Multi-document and extended-conversation assistants where inference latency and KV-cache memory are the binding constraints.
  • Retrieval-heavy workflows over tables and numerical reports, which the training data explicitly covers through FinQA and TAT-QA sources.
  • Deployments that need compressed visual input while still requiring general multimodal ability (charts, figures, diagrams), since MMMU and MME are preserved or improved.
• Industry relevance: The reported 2.79 times online end-to-end speedup and roughly 2.9 times input compression at 72 DPI speak directly to serving cost. The finding that tool access helps only after policy learning — untrained tool use degrades every aggregate — is a practical caution for anyone bolting an enhancement interface onto an existing model. The paper also ties an engineering detail, font choice, to downstream accuracy, which matters for anyone building a rendering pipeline.

Future Directions

1. Pushing compression further. The paper reports that at 48 and 60 DPI extra-observation costs rise and savings shrink, while above 72 DPI prompt length grows for modest gains. Whether lower initial resolutions can be made to work with better cropping policies is left open.
2. Extending evaluation beyond the reported suite. Results are reported for LongBench, MRCR, VTCBench, RULER v1/v2, MMMU, MME and other general multimodal benchmarks; the paper does not report results on other domains or languages, so generalization there is not established.
3. Reducing dependence on synthetic annotations. REL-CoT is generated and verified by three separate models. Whether similar adaptive reading can be learned with less annotation, or with weaker verifiers, is not addressed.
4. Understanding the reasoning trade-off. VTCBench Reasoning remains weaker than the Retrieval and Memory results and is the one aggregate where REL-SFT initialization does not help over direct GRPO. What limits reasoning over compressed visual context is not resolved.

Target Audience

Researchers and engineers working on efficient long-context inference, vision-language models, and multimodal document understanding will benefit most. It is also relevant to practitioners building document QA or agentic retrieval systems who need concrete numbers on token savings and latency, and to anyone studying reinforcement learning for tool-augmented multimodal models — particularly the GRPO reward design and the finding that policy learning, not the interface itself, is what makes tool use pay off. Readers without background in VLM training and RL fine-tuning will find the training sections dense, but the framing of the compression–performance trade-off in the introduction and the benchmark tables are accessible.

Authors’ abstract

Long-context reasoning in large language models incurs substantial computation and memory costs. Visual text compression (VTC) reduces input length by rendering text as images, but fixed-resolution rendering creates a compression-performance trade-off: low DPI saves tokens at the expense of legibility, whereas high DPI spends tokens on irrelevant content. We introduce FocusVTC, which breaks this trade-off through adaptive resolution while preserving general multimodal capabilities. It combines compressed low-DPI global views with selective region enhancement, integrating enhanced views into ongoing reasoning. We construct 29.4K high-quality Reasoning-Evidence Localization (REL) chain-of-thought examples (REL-CoT) that link reasoning traces to page indices and bounding boxes. Multi-resolution REL supervised fine-tuning (REL-SFT) teaches the model to localize relevant regions, and Group Relative Policy Optimization learns when to enhance resolution and how to use the resulting observations, without a separate continual-pretraining stage. At 72 DPI on RULER v1, FocusVTC scores 87.4 at $2.9\times$ input compression, including tool observations, versus 57.5 for Glyph at $3.0\times$ input compression. It surpasses its text-input backbone on LongBench (56.40 versus 55.86), improves the MRCR macro-average by 13.91 points, and achieves a 51.19 macro-average on VTCBench. The MRCR latency evaluation also shows a $2.79\times$ online end-to-end speedup over Text. General multimodal capabilities are preserved, with MMMU increasing from 65.12 to 66.73 and MME from 2424.02 to 2457.62.