Typhoon-S: Minimal Open Post-Training for Sovereign Large Language Models
Not on the map. Why?
Abstract (arXiv)
Large language models (LLMs) have progressed rapidly; however, most state-of-the-art models are trained and evaluated primarily in high-resource languages such as English and Chinese, and are often developed by a small number of organizations with access to large-scale compute and data. This gatekeeping creates a practical barrier for sovereign settings in which a regional- or national-scale institution or domain owner must retain control and understanding of model weights, training data, and deployment while operating under limited resources and strict transparency constraints. To this end, we identify two core requirements: (1) adoptability, the ability to transform a base model into a general-purpose assistant, and (2) sovereign capability, the ability to perform high-stakes, region-specific tasks (e.g., legal reasoning in local languages and cultural knowledge). We investigate whether these requirements can be achieved without scaling massive instruction corpora or relying on complex preference tuning pipelines and large-scale reinforcement fine-tuning (RFT). We present Typhoon S, a minimal and open post-training recipe that combines supervised fine-tuning, on-policy distillation, and small-scale RFT. Using Thai as a representative case study, we demonstrate that our approach transforms both sovereign-adapted and general-purpose base models into instruction-tuned models with strong general performance. We further show that small-scale RFT with InK-GRPO -- an extension of GRPO that augments the GRPO loss with a next-word prediction loss -- improves Thai legal reasoning and Thai-specific knowledge while preserving general capabilities. Our results suggest that a carefully designed post-training strategy can reduce the required scale of instruction data and computation, providing a practical path toward high-quality sovereign LLMs under academic-scale resources.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 30 of its 44 references that are in the map. First posted on arXiv 26 Jan 2026.
Builds on
30 of 44 references.
References · 30
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2025Qwen3 Technical Reportcited by 8,195
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
- 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
- 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
- 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
- 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 840
- 2016Sequence-Level Knowledge Distillationcited by 761
- 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 655
18 more
- 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 598
- 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 486
- 2025Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretrainingcited by 123
- 2024How Do Large Language Models Acquire Factual Knowledge During Pretraining?cited by 86
- 2025Retaining by Doing: The Role of On-Policy Data in Mitigating Forgettingcited by 82
- 2025Do Multilingual LLMs Think In English?cited by 75
- 2024Self-play with Execution Feedback: Improving Instruction-following Capabilities of Large Language Modelscited by 73
- 2025MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluationcited by 73
- 2025TOUCAN: Synthesizing 1.5M Tool-Agentic Data from Real-World MCP Environmentscited by 61
- 2025The Path Not Taken: RLVR Provably Learns Off the Principalscited by 52
- 2025From f(x) and g(x) to f(g(x)): LLMs Learn New Skills in RL by Composing Old Onescited by 40
- 2024MIRAGE-Bench: Automatic Multilingual Benchmark Arena for Retrieval-Augmented Generation Systemscited by 14
- 2025Adapting Language-Specific LLMs to a Reasoning Model in One Day via Model Merging -- An Open Recipecited by 9
- 2024OpenThaiGPT 1.5: A Thai-Centric Open Source Large Language Modelcited by 8
- 2025Typhoon T1: An Open Thai Reasoning Modelcited by 3
- 2025OpenThaiGPT 1.6 and R1: Thai-Centric Open Source and Reasoning Large Language Modelscited by 2
- 2025Prior Prompt Engineering for Reinforcement Fine-Tuningcited by 1
- 2025On the Robustness of Answer Formats in Medical Reasoning Modelscited by 1
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Large Language Model Post-Training: A Unified View of Off-Policy and On-Policy Learning
- 2025LLM Post-Training: A Deep Dive into Reasoning Large Language Models
- 2025StatLLaMA: Multi-Stage training for domain-optimized statistical large language models
- 2024Responsible Multilingual Large Language Models: A Survey of Development, Applications, and Societal Impact
- 2024Sailor: Open Language Models for South-East Asia
- 2025Reasoning Beyond Limits: Advances and Open Problems for LLMs
- 2024Pangea: A Fully Open Multilingual Multimodal LLM for 39 Languages
- 2025Fluent Alignment with Disfluent Judges: Post-training for Lower-resource Languages