mlsanityData to 7 Oct 2026

Typhoon: Thai Large Language Models

arXiv 2312.13951 ↗First posted 21 Dec 2023cs.CL cs.AIcited by 28

Not on the map. Why?

Abstract (arXiv)

Typhoon is a series of Thai large language models (LLMs) developed specifically for the Thai language. This technical report presents challenges and insights in developing Thai LLMs, including data preparation, pretraining, instruction-tuning, and evaluation. As one of the challenges of low-resource languages is the amount of pretraining data, we apply continual training to transfer existing world knowledge from a strong LLM. To evaluate the Thai knowledge encapsulated in each model from the pretraining stage, we develop ThaiExam, a benchmark based on examinations for high-school students and investment professionals in Thailand. In addition, we fine-tune Typhoon to follow Thai instructions, and we evaluate instruction-tuned models on Thai instruction datasets as well as translation, summarization, and question-answering tasks. Experimental results on a suite of Thai benchmarks show that Typhoon outperforms all open-source Thai language models, and its performance is on par with GPT-3.5 in Thai while having only 7 billion parameters and being 2.62 times more efficient in tokenizing Thai text.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 43 of its 58 references that are in the map. First posted on arXiv 21 Dec 2023.

Builds on

43 of 58 references.

References · 43

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  9. 2023Mistral 7Bcited by 3,065
  10. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  11. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  12. 2020Learning to summarize from human feedbackcited by 2,562
31 more
  1. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  2. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  3. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  4. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  5. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  6. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  7. 2023BloombergGPT: A Large Language Model for Financecited by 923
  8. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 913
  9. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  10. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  11. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 825
  12. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 622
  13. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 615
  14. 2019On the Cross-lingual Transferability of Monolingual Representationscited by 575
  15. 2019CamemBERT: a Tasty French Language Modelcited by 368
  16. 2021XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languagescited by 346
  17. 2020XCOPA: A Multilingual Dataset for Causal Commonsense Reasoningcited by 330
  18. 2023MEGA: Multilingual Evaluation of Generative AIcited by 291
  19. 2021Few-shot Learning with Multilingual Language Modelscited by 278
  20. 2020PhoBERT: Pre-trained language models for Vietnamesecited by 148
  21. 2020A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languagescited by 131
  22. 2023M3Exam: A Multilingual, Multimodal, Multilevel Benchmark for Examining Large Language Modelscited by 109
  23. 2023SeaLLMs -- Large Language Models for Southeast Asiacited by 104
  24. 2023Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Modelscited by 68
  25. 2023LLM Comparative Assessment: Zero-shot NLG Evaluation through Pairwise Comparisons using Large Language Modelscited by 66
  26. 2021CrossSum: Beyond English-Centric Cross-Lingual Summarization for 1,500+ Language Pairscited by 39
  27. 2023Efficiently Adapting Pretrained Language Models To New Languagescited by 28
  28. 2021WangchanBERTa: Pretraining transformer-based Thai Language Modelscited by 25
  29. 2022Cedille: A large autoregressive French language modelcited by 20
  30. 2022IT5: Text-to-text Pretraining for Italian Language Understanding and Generationcited by 20
  31. 2023BHASA: A Holistic Southeast Asian Linguistic and Cultural Evaluation Suite for Large Language Modelscited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 18; 2025: 8; 2026: 2.

28 papers.

Sort:

By area:

  1. 2024Survey of Cultural Awareness in Language Models: Text and Beyondcited by 101
  2. 2024On the Societal Impact of Open Foundation Modelscited by 64
  3. 2024CultureLLM: Incorporating Cultural Differences into Large Language Modelscited by 49
  4. 2025CultureVLM: Characterizing and Improving Cultural Understanding of Vision-Language Models for over 100 Countriescited by 40
  5. 2024CulturePark: Boosting Cross-cultural Understanding in Large Language Modelscited by 24
  6. 2024How Can We Effectively Expand the Vocabulary of LLMs with 0.01GB of Target Language Text?cited by 22
  7. 2024The Model Openness Framework: Promoting Completeness and Openness for Reproducibility, Transparency, and Usability in Artificial Intelligencecited by 21
  8. 2024Sailor: Open Language Models for South-East Asiacited by 19
  9. 2024The AI Community Building the Future? A Quantitative Analysis of Development Activity on Hugging Face Hubcited by 18
  10. 2024Typhoon 2: A Family of Open Text and Multimodal Thai Large Language Modelscited by 12
  11. 2024Sabi'a-2: A New Generation of Portuguese Large Language Modelscited by 10
  12. 2024The Future of Open Human Feedbackcited by 10
16 more
  1. 2024Enhancing Low-Resource Language and Instruction Following Capabilities of Audio Language Modelscited by 10
  2. 2024OpenThaiGPT 1.5: A Thai-Centric Open Source Large Language Modelcited by 8
  3. 2025Building High-Quality Datasets for Portuguese LLMs: From Common Crawl Snapshots to Industrial-Grade Corporacited by 7
  4. 2024Adapting Chat Language Models Using Only Target Unlabeled Language Datacited by 6
  5. 2025CAReDiO: Cultural Alignment via Representativeness and Distinctiveness Guided Data Optimizationcited by 5
  6. 2025Typhoon T1: An Open Thai Reasoning Modelcited by 3
  7. 2025OpenThaiGPT 1.6 and R1: Thai-Centric Open Source and Reasoning Large Language Modelscited by 2
  8. 2024Thai Winograd Schemas: A Benchmark for Thai Commonsense Reasoningcited by 1
  9. 2024Eir: Thai Medical Large Language Modelscited by 1
  10. 2025Vocabulary Customization for Efficient Domain-Specific LLM Deploymentcited by 1
  11. 2024Bailong: Bilingual Transfer Learning based on QLoRA and Zip-tie Embeddingcited by 0
  12. 2024GECKO: Generative Language Model for English, Code and Koreancited by 0
  13. 2025JAI-1: A Thai-Centric Large Language Modelcited by 0
  14. 2025Fluent Alignment with Disfluent Judges: Post-training for Lower-resource Languagescited by 0
  15. 2026TSWAP: A Multilingual Retrieval-Augmented Thai Wellness Advisorcited by 0
  16. 2026SEA-LION-v4.8: A Technical Reportcited by 0