Typhoon-S: Minimal Open Post-Training for Sovereign Large Language Models

AI summary

Overview

• Research area: Natural language processing; post-training of large language models, with a focus on "sovereign" (region- or domain-specific) model development.
• Technical level: Advanced (assumes familiarity with supervised fine-tuning, knowledge distillation, GRPO/reinforcement fine-tuning, and agentic tool use).
• Scope: A technical report presenting Typhoon-S, a minimal, open post-training recipe combining supervised fine-tuning, on-policy distillation, and small-scale reinforcement fine-tuning, validated with Thai as the representative sovereign language.

What This Paper Is About

State-of-the-art large language models are built mainly by a small number of organizations with large compute budgets and are trained predominantly on English- and Chinese-centric data, which makes it hard for national or regional institutions to build models they fully control and understand. The paper asks which post-training strategy can produce competitive performance under academic-level resource constraints, splitting the problem into adoptability (turning a base model into a general-purpose assistant) and sovereign capability (performing high-stakes, region-specific tasks such as local legal reasoning and cultural knowledge). It uses Thai as a case study and reports a minimal, openly described recipe rather than a large-scale pipeline.

Key Contributions

1. A framing of two complementary requirements for sovereign post-training: adoptability as a general-purpose assistant and sovereign capability on regional- or culturally-specific tasks.
2. A minimal base-to-instruct recipe for adoptability that combines lightweight SFT with on-policy distillation (OPD), using open-source English instruction data (Tulu 3, Toucan) together with a small amount of target-language (Thai) data.
3. InK-GRPO (Injected Knowledge GRPO), an extension of GRPO that augments the GRPO loss with a cross-entropy next-token-prediction loss applied stochastically, intended to optimize domain performance while teaching new knowledge in parallel.
4. An agentic RFT setup where the model uses search and read tools against a controlled retrieval-augmented environment during both training and inference, with final-answer accuracy as the reward.

Main Findings

• SFT alone is insufficient. Compared against an SFT-only model, the full recipe (SFT+OPD) raised the average score by +6.49 points (37.45 → 43.94). The SFT-only model dropped sharply on Thai code-switching (CS: 65.4 vs. 93.4 for SFT+OPD) and scored zero on HPQA in both English and Thai. Both SFT-only (37.45) and SFT+OPD (43.94) remained below the Qwen3 Instruct baseline average of 48.07 on the full suite.
• Full-logits distillation improves sampling robustness, but is not always necessary. Full-logits OPD achieved a higher average than Top-K OPD (43.94 vs. 42.81), with the largest gap on Thai code-switching (CS: 93.4 vs. 69.8). Top-K performed comparably or slightly better on some single-correct-answer tasks such as MATH500 and HotpotQA.
• Target-language data is essential at the SFT stage. Removing Thai data degraded Thai chat (MTB TH: 5.67 → 4.36), Thai instruction following (IFE TH: 73.35 → 57.44), and code-switching (CS: 65.4 → 34.4), dropping the SFT average to 33.07.
• Target-language data matters less at the OPD stage and is more targeted. Without Thai data at OPD, the average was 42.02 vs. 43.94. Thai data mainly improved Thai-native tasks: Thai chat (6.28 → 6.44), OpenThaiEval (59.32 → 61.19), code-switching (80.8 → 93.4), and Thai instruction following (72.62 → 75.98), while translated or language-agnostic tasks such as MATH500 and MMLU Pro X changed little.
• The recipe transfers to a sovereignty-adapted base model. Starting from ThaiLLM-8B (Qwen3-8B-Base further pretrained on 64B tokens of Thai corpus), the resulting Typhoon-S-8B surpassed Qwen3-8B on a Thai-only suite: average 71.20 vs. 66.66, including Thai chat (7.89 vs. 7.08), code-switching (96.60 vs. 95.40), OpenThaiEval (67.06 vs. 63.66), and HPQA Thai (37.00 vs. 23.00). It trailed on Thai instruction following (76.45 vs. 80.47).
• On the full English+Thai suite, a gap remains in hard knowledge, math, and code. Typhoon-S-8B trailed Qwen3-8B in GPQA, MMLU, MATH500, and LiveCodeBench. The paper's running text reports overall averages of 51.88 vs. 54.72, while Table 7 in the provided content lists 49.99 vs. 54.02; these two figures are inconsistent in the supplied text.
• Sovereign capability results are only partially available. The abstract states that small-scale RFT with InK-GRPO improves Thai legal reasoning and Thai-specific knowledge while preserving general capabilities, with particularly strong improvements in Thai legal reasoning. The detailed sovereign-capability results are not included in the provided (truncated) content, so specific benchmark numbers for that stage are not reported here.
• Compute footprint. The paper reports approximately two days of 8-GPU training for an 8B model for adoptability, and one day of 4-GPU training for sovereign capability. The 4B-scale experiments ran on 4×H100 GPUs and completed in under two days; the 8B-scale experiments ran on 8×H100 GPUs over approximately two days. The 8B OPD stage itself was trained on 4×H100 GPUs.

Methodology in Plain English

The work proceeds in two separate experimental tracks to avoid cross-contamination between general ability and domain-specific ability.

For adoptability, the team built a two-stage pipeline:

1. Supervised fine-tuning (SFT) on a mixed corpus of 340k examples: 200k from Tulu 3 SFT (general instructions, sampled from a dataset of over one million examples), 100k from Toucan Tool (tool use), and 40k Thai AutoIF examples. SFT used AdamW at a learning rate of 2 × 10⁻⁵, batch size 32, sequence packing up to 16,384 tokens, for two epochs.
2. On-policy distillation (OPD) using Generalized Knowledge Distillation. At each step, with probability λ = 0.25 the student generates its own output (on-policy data); otherwise a sequence is drawn from a reference dataset. The student is trained to match the teacher's token-level distribution with a forward KL divergence. OPD used AdamW at 1 × 10⁻⁶ for a single epoch on a 160k-example corpus (Tulu 3 subset 100k, Thai AutoIF 40k, Toucan Tool subset 20k). Both full-logits and Top-K variants were compared; full-logits was adopted for the remaining experiments.

The Thai data was constructed by translating WildChat real-user prompts into Thai, aggregating prompts from WangchanThaiInstruct, Han, and Typhoon Instruct, generating responses with Qwen3 235B A22B Instruct (2507) under the AutoIF framework, and discarding responses scoring below 7. Augmentations randomly translated constraints between English and Thai and randomly placed constraints in the system or user message.

To make full-logits distillation practical on a single node, the authors built a lightweight framework on HuggingFace Transformers and TRL with dynamic model swapping (inactive models offloaded to RAM), FSDP with CPU offloading, vLLM as the inference backend, and hybrid scheduling.

For sovereign capability, the authors extend GRPO in two ways. InK-GRPO adds a cross-entropy next-token-prediction loss computed on a separate in-domain text corpus, applied stochastically according to a Bernoulli(ρ) draw, with λ controlling its relative weight alongside the GRPO loss. Agentic RFT trains the model to interleave reasoning with tool calls (search for semantic retrieval returning document identifiers, read for full document contents) inside a partially observable RAG environment, applying GRPO over whole interaction trajectories with tool outputs masked from gradient computation and reward based on final-answer accuracy.

Evaluation covers chat (MT-Bench, Thai MT-Bench), instruction following (IFEval, scb10x/ifeval-th), Thai code-switching robustness, knowledge (GPQA Diamond; MMLU Pro X Thai; OpenThaiEval), math (MATH500, 500 English problems plus a Thai-translated version totaling 1,000), code (LiveCodeBench release_v3, 612 problems, pass@1), and tool use/agentic reasoning (BFCL v4 single- and multi-tool scenarios, 4,441 problems; HotpotQA medium subset subsampled to 100 questions with a ReAct-style agent connected to the Wikipedia API). Accuracy is the primary metric under greedy decoding, with the average computed as an unweighted mean across benchmarks and MT-Bench multiplied by 10.

Why This Matters

The paper argues that current state-of-the-art development is gated by resources: even "fully open" efforts such as OLMo 3 used a cluster of 1,024 H100 GPUs for several months, totaling $2.75M. If a sovereign model can be produced with a two-day, 8-GPU run for general capability and a one-day, 4-GPU run for domain capability, that substantially lowers the barrier for national institutions, public-sector bodies, and smaller research groups.

Real-world applications:

• Government and public-sector assistants that must operate in a national language under strict transparency requirements while keeping control of weights and data.
• Legal and regulatory work, the specific domain targeted by the InK-GRPO and agentic RFT experiments (Thai legal reasoning) and by the released Typhoon-S-4B-Legal-Agent model.
• Cultural and regional knowledge services, such as education and exam-style question answering built on native-language benchmarks like OpenThaiEval and the M3/M6 exams.
• Tool-using local agents that retrieve from an in-domain document corpus (via search and read) rather than relying on knowledge baked into model weights.

Industry relevance: the recipe is released as open collections, models (Typhoon-S-8B-Instruct, Typhoon-S-4B-Legal-Agent), datasets (Typhoon-S-Instruct-Dataset, Typhoon-S-Sovereign-Capability-Dataset), and code, giving organizations a reproducible starting point for domain adaptation without building a large-scale post-training pipeline.

Future Directions

• Whether local knowledge alone drives gains on agentic tasks. The authors observe improved agentic performance when their method is applied to a sovereignty-focused model and explicitly leave open whether local knowledge alone is sufficient to improve performance in low-resource languages under unbiased post-training.
• Closing the gap on hard knowledge, math, and code. Typhoon-S-8B still trails Qwen3-8B on GPQA, MMLU, MATH500, and LiveCodeBench; how to recover these without sacrificing Thai-native strength is unresolved.
• Better evaluation of royalty-focused models. The authors note that several translated benchmarks may be biased toward highly multilingual models such as Qwen3-8B (119 languages), suggesting a need for benchmarks that fairly measure sovereign performance.
• Extending the sovereign-capability stage. The report introduces InK-GRPO and agentic RFT but the provided content does not include their full experimental results, leaving the tuning of the stochastic CE injection (ρ and λ) and the agentic environment design as open questions.

Target Audience

Researchers and engineers working on low-resource or region-specific language model adaptation, national AI initiatives and public-sector AI teams with limited compute, practitioners interested in practical distillation and reinforcement fine-tuning recipes, and anyone studying how to combine SFT, on-policy distillation, and GRPO-style training under bounded GPU budgets.

Authors’ abstract

Large language models (LLMs) have progressed rapidly; however, most state-of-the-art models are trained and evaluated primarily in high-resource languages such as English and Chinese, and are often developed by a small number of organizations with access to large-scale compute and data. This gatekeeping creates a practical barrier for sovereign settings in which a regional- or national-scale institution or domain owner must retain control and understanding of model weights, training data, and deployment while operating under limited resources and strict transparency constraints. To this end, we identify two core requirements: (1) adoptability, the ability to transform a base model into a general-purpose assistant, and (2) sovereign capability, the ability to perform high-stakes, region-specific tasks (e.g., legal reasoning in local languages and cultural knowledge). We investigate whether these requirements can be achieved without scaling massive instruction corpora or relying on complex preference tuning pipelines and large-scale reinforcement fine-tuning (RFT). We present Typhoon S, a minimal and open post-training recipe that combines supervised fine-tuning, on-policy distillation, and small-scale RFT. Using Thai as a representative case study, we demonstrate that our approach transforms both sovereign-adapted and general-purpose base models into instruction-tuned models with strong general performance. We further show that small-scale RFT with InK-GRPO -- an extension of GRPO that augments the GRPO loss with a next-word prediction loss -- improves Thai legal reasoning and Thai-specific knowledge while preserving general capabilities. Our results suggest that a carefully designed post-training strategy can reduce the required scale of instruction data and computation, providing a practical path toward high-quality sovereign LLMs under academic-scale resources.