# NVIDIA Maps a 2.8 Trillion-Token Path for Indic AI

Published 2026-10-16 · AI.info Team · AI.info

> NVIDIA and AI4Bharat announced a 2.8 trillion-token corpus for 22 Indian languages, with native-script data in all 22 and romanized text in 12. The Hugging Face post details the translation and quality-check pipeline and reports benchmark gains from a smaller mixed-data training run.

On October 16, [NVIDIA](https://ai.info/companies/nvidia) and AI4Bharat announced a planned release of about 2.8 trillion tokens of training text across 22 Indian languages, alongside a GPU-based recipe for generating the data. The post, published on [Hugging Face](https://huggingface.co/blog/nvidia/scaling-indic-pretraining), describes a corpus built largely by translating English material—not a collection of 2.8 trillion tokens originally written in Indian languages. NVIDIA says the dataset is still “releasing soon”; the generation code and workflow are being shared separately.

## A 2.8 trillion-token corpus in 34 configurations

Named Nemotron-pretraining-Indic-v1, the corpus contains 2,601.0 billion native-script tokens and 224.6 billion romanized tokens, counted with the Gemma-4 tokenizer. It covers native-script text for all 22 languages and Latin-script versions for 12, creating 34 language-and-script configurations. The 22 languages are those listed in India’s Constitution, the post says.

The romanized material is not English: it is Indic-language text written in Latin letters. The corpus also includes translated records and script variants derived from shared English sources, so the token total does not represent unique documents or a training run’s token budget. The dataset draws on 15 Nemotron-CC-v2 subsets covering web text, questions and answers, mathematics, code and specialized material.

## Translation, scoring and repair

For 12 languages, the workflow uses Gemma-4-31B-it to translate full English documents, assess each translation against its source and help repair outputs that score below the acceptance threshold. The rubric checks meaning, fluency, terminology, style and formatting; accepted translations must score at least 75 out of 100, a rubric score rather than an accuracy percentage. Prompts tell the model to preserve elements such as equations, tables, code and URLs.

Accepted native-script translations also supply the starting point for romanized versions, which receive a separate check against the native-script text. The other 10 languages follow a different path: Bodhan AI’s Indic Translate performs translation, and Gemma-4-31B-it assesses the results. The repair process recovered 438.4 billion tokens across native-script and romanized data, already included in the corpus total.

## The reported gains came from a smaller training mixture

The announced corpus is much larger than the amount used in the reported Nemotron 3.5 Super VL continued-pretraining stage. That run had a 100-billion-token budget, with 10 billion tokens allocated to a mixture of synthetic and naturally authored Indic text; the remaining 90 billion retained the earlier training mixture’s proportions. The model improved by 4.98 to 9.82 percentage points across five reported Indic benchmarks, according to NVIDIA’s technical report.

The results do not isolate the synthetic corpus’s contribution: they measure the combined stage using synthetic and natural Indic material. The report also identifies the benchmarks as comparisons between checkpoints immediately before and after continued pretraining, not a direct test of the full 2.8-trillion-token corpus. A study of the complete corpus, including repaired text and the 10-language workflow, remains to be done, NVIDIA’s blog says.

## Data volume does not replace native writing

The authors frame the project as a response to a shortage of training material. “We do not have enough data for Indian languages,” Mitesh Khapra, head of AI4Bharat, writes in the post. The translated corpus can broaden the supply of material for technical subjects, but it cannot ensure coverage of local knowledge or culture, and the authors recommend combining it with appropriately sourced, naturally written Indic text.

Quality checks are primarily automated, with sampled human review whose coverage varies by language. In the 12-language workflow, the same model translates and judges the text; NVIDIA cautions that retained outputs can still contain errors. The dataset’s token count is large, but its practical value will depend on the forthcoming release, the language-specific data, and how well the synthetic text holds up when researchers evaluate it alongside native sources.

## Sources

1. Original source · Blog post [Hugging Face](https://huggingface.co/blog/nvidia/scaling-indic-pretraining) huggingface.co

Other pages read

2. [NVIDIA](https://research.nvidia.com/labs/nemotron/files/NVIDIA-Nemotron-3.5-Super-and-Lightning-Technical-Report.pdf) research.nvidia.com

## Companies in this story

- [Nvidia](https://ai.info/companies/nvidia)

## Related stories

- [One AI training step cut review diversity by 11%](https://ai.info/news/one-ai-training-step-cut-review-diversity-by-11-percent)
- [NVIDIA Pledges $250 Million in UK AI Safety Compute](https://ai.info/news/nvidia-250-million-uk-agentic-safety-compute)
- [NVIDIA and UW Open RFD4-Proteina for Protein Design](https://ai.info/news/nvidia-uw-open-rfd4-proteina-protein-design)
- [NVIDIA Opens Nemotron’s IOI and IMO Gold Recipes](https://ai.info/news/nvidia-opens-nemotron-ioi-imo-gold-recipes)
- [NVIDIA and Partners Release Protein Predictions for 2,812 Viral Proteomes](https://ai.info/news/nvidia-partners-viral-protein-predictions-2812-proteomes)

## Explore

[More articles](https://ai.info/news)

---

Canonical: https://ai.info/news/nvidia-2-8-trillion-token-indic-pretraining-pipeline
