The Pulse
Dharma-AI Says VocabX Cut Grader Costs 77%
Dharma-AI says its VocabX vocabulary method cut measured inference costs by 77% in a Portuguese essay grader running Qwen3-4B. The company reports shorter responses and a higher quality score, while noting that its paper is still under revi
AI.info Team · Written with AI · fact-checked ·
“In our grader, that cut measured inference cost by 77%, with the same base model and the same training recipe.”
Bernardo França de Aguiar, Gabriel Pimenta de Freitas Cardoso, Bruno Duarte and Francisco de Almeida Rocha Alves, Dharma-AI
Dharma-AI says its VocabX method lowered measured inference costs in a production Portuguese essay grader built on Qwen3-4B. The team published the result on October 21, 2026, in a Hugging Face post previewing a paper that remains under review. Its central change was not a new base model, but a vocabulary tailored to the grader’s language and recurring JSON output.
Qwen3-4B’s vocabulary was built for broader work
The grader reads an essay, its prompt and supporting texts, then returns a JSON assessment with a grade and comment for each competency. Dharma-AI says the system’s original Qwen3-4B tokenizer contained 151,669 entries, many of which the Portuguese-only task rarely used. The team identifies two sources of waste: repeated phrases and JSON structures took multiple tokens to generate, and the model scored the full vocabulary at every generation step.
VocabX changes tokenization at both the sub-word and multi-word levels. It merges domain-specific word pieces and adds “macro” tokens for repeated spans, including full lines and JSON structures. The team says it selects tokens according to their estimated serving cost rather than frequency alone, and controls vocabulary size through a fixed budget for macros. In the grader, the output layer fell from about 152,000 rows to about 65,000.
Seven versions ran on the same test machine
Dharma-AI compared seven grader versions using the same base model, data, training recipe and random seed. Each version handled the same 500 requests on a machine with an RTX 5090 GPU and the vLLM inference server; the team calculated cost from measured runtime. Average response length fell from 350 tokens to 101, while the reported cost reduction reached 77.0%.
The team also reports a higher bench score, rising from 0.703 to 0.810. Its score combines grading accuracy with the accuracy of screening out off-topic or rule-breaking essays. The result reflects the tested grader and setup; the post cautions that generated-token costs vary with hardware, batch size, context length and other conditions.
An early design fused each grade with surrounding JSON into a single macro token. That version was more compact, but its bench score dropped to 0.689, below the production grader’s 0.703. Giving the grade its own token added two tokens per response, raised the score to 0.810 and produced a larger cost reduction: 77.0%, compared with 73.3% for the fused version.
Broader tests show estimated savings, not a second 77%
The team also compared VocabX with three reference methods across five public domains—contracts, medical papers, scientific papers, news and Python code—and three model families: Qwen3, Llama-3.1 and Gemma-3. In all 15 combinations, VocabX compressed text more than the strongest reference method, by 1.2 to 9.3 percentage points. Its pruned version also led on estimated inference cost in all 15 comparisons, with average margins of 10.2 points on Qwen3, 7.5 on Llama-3.1 and 13.6 on Gemma-3.
Those cross-domain cost figures are estimates, not the measured runtime result from the production grader. The post says the full method, experiment tables and discarded interventions will appear in the paper, which Dharma-AI describes as still under review and subject to change. For now, the 77% result is tied to one deployed grading system and one specified test setup—not a cost guarantee for other models or workloads.