mlsanityData to 14 Oct 2026

SeaLLMs -- Large Language Models for Southeast Asia

arXiv 2312.00738 ↗First posted 1 Dec 2023cs.CLcited by 104

Not on the map. Why?

Abstract (arXiv)

Despite the remarkable achievements of large language models (LLMs) in various tasks, there remains a linguistic bias that favors high-resource languages, such as English, often at the expense of low-resource and regional languages. To address this imbalance, we introduce SeaLLMs, an innovative series of language models that specifically focuses on Southeast Asian (SEA) languages. SeaLLMs are built upon the Llama-2 model and further advanced through continued pre-training with an extended vocabulary, specialized instruction and alignment tuning to better capture the intricacies of regional languages. This allows them to respect and reflect local cultural norms, customs, stylistic preferences, and legal considerations. Our comprehensive evaluation demonstrates that SeaLLM-13b models exhibit superior performance across a wide spectrum of linguistic tasks and assistant-style instruction-following capabilities relative to comparable open-source models. Moreover, they outperform ChatGPT-3.5 in non-Latin languages, such as Thai, Khmer, Lao, and Burmese, by large margins while remaining lightweight and cost-effective to operate.

Key facts. Cited by 104 papers in the map (data through 14 Oct 2026). Builds on 33 of its 44 references that are in the map. First posted on arXiv 1 Dec 2023.

Builds on

33 of 44 references.

References · 33

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  7. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  8. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  9. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  10. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  11. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  12. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
21 more
  1. 2023Qwen Technical Reportcited by 3,344
  2. 2023Mistral 7Bcited by 3,164
  3. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  4. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  5. 2020Learning to summarize from human feedbackcited by 2,588
  6. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  7. 2016Enriching Word Vectors with Subword Informationcited by 1,583
  8. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  9. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  10. 2019On the Cross-lingual Transferability of Monolingual Representationscited by 578
  11. 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 548
  12. 2023Zephyr: Direct Distillation of LM Alignmentcited by 506
  13. 2023Orca: Progressive Learning from Complex Explanation Traces of GPT-4cited by 379
  14. 2023MEGA: Multilingual Evaluation of Generative AIcited by 293
  15. 2023Multilingual Jailbreak Challenges in Large Language Modelscited by 240
  16. 2023ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learningcited by 206
  17. 2023Platypus: Quick, Cheap, and Powerful Refinement of LLMscited by 150
  18. 2023Benchmarking Cognitive Biases in Large Language Models as Evaluatorscited by 150
  19. 2023M3Exam: A Multilingual, Multimodal, Multilevel Benchmark for Examining Large Language Modelscited by 110
  20. 2023PolyLM: An Open Source Polyglot Large Language Modelcited by 64
  21. 2023Democratizing LLMs for Low-Resource Languages by Leveraging their English Dominant Abilities with Linguistically-Diverse Promptscited by 45

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 48; 2025: 34; 2026: 20.

104 papers.

Sort:

By area:

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2024BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languagescited by 173
  3. 2024How do Large Language Models Handle Multilingualism?cited by 150
  4. 2024Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Modelscited by 126
  5. 2024Survey of Cultural Awareness in Language Models: Text and Beyondcited by 103
  6. 2024LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shiftscited by 86
  7. 2024CultureLLM: Incorporating Cultural Differences into Large Language Modelscited by 50
  8. 2024SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languagescited by 43
  9. 2025CultureVLM: Characterizing and Improving Cultural Understanding of Vision-Language Models for over 100 Countriescited by 42
  10. 2024IndoCulture: Exploring Geographically-Influenced Cultural Commonsense Reasoning Across Eleven Indonesian Provincescited by 39
  11. 2024SeaLLMs 3: Open Foundation and Chat Multilingual Large Language Models for Southeast Asian Languagescited by 36
  12. 2024Is Translation All You Need? A Study on Solving Multilingual Tasks with Large Language Modelscited by 31
38 more
  1. 2024Self-Pluralising Culture Alignment for Large Language Modelscited by 31
  2. 2023Typhoon: Thai Large Language Modelscited by 30
  3. 2024RomanSetu: Efficiently unlocking multilingual capabilities of Large Language Models via Romanizationcited by 29
  4. 2024VarBench: Robust Language Model Benchmarking Through Dynamic Variable Perturbationcited by 29
  5. 2024CulturePark: Boosting Cross-cultural Understanding in Large Language Modelscited by 24
  6. 2024Preference Tuning with Human Feedback on Language, Speech, and Vision Tasks: A Surveycited by 23
  7. 2024How Can We Effectively Expand the Vocabulary of LLMs with 0.01GB of Target Language Text?cited by 22
  8. 2025SEA-LION: Southeast Asian Languages in One Networkcited by 22
  9. 2024High-Dimension Human Value Representation in Large Language Modelscited by 20
  10. 2024An Empirical Comparison of Vocabulary Expansion and Initialization Approaches for Language Modelscited by 20
  11. 2024Sailor: Open Language Models for South-East Asiacited by 19
  12. 2024Unlocking the Potential of Model Merging for Low-Resource Languagescited by 19
  13. 2025SEA-HELM: Southeast Asian Holistic Evaluation of Language Modelscited by 19
  14. 2024Cendol: Open Instruction-tuned Generative Large Language Models for Indonesian Languagescited by 18
  15. 2024Multilingual Large Language Models: A Systematic Surveycited by 18
  16. 2025Sailor2: Sailing in South-East Asia with Inclusive Multilingual LLMscited by 18
  17. 2024MKGL: Mastery of a Three-Word Languagecited by 17
  18. 2025Babel: Open Multilingual Large Language Models Serving Over 90% of Global Speakerscited by 16
  19. 2025Disentangling Language and Culture for Evaluating Multilingual Large Language Modelscited by 16
  20. 2023PersianLLaMA: Towards Building First Persian Large Language Modelcited by 14
  21. 2025SeaExam and SeaBench: Benchmarking LLMs with Local Multilingual Questions in Southeast Asiacited by 13
  22. 2025Less, but Better: Efficient Multilingual Expansion for LLMs via Layer-wise Mixture-of-Expertscited by 12
  23. 2025The Landscape of Arabic Large Language Models (ALLMs): A New Era for Arabic Language Technologycited by 12
  24. 2026AfriqueLLM: How Data Mixing and Model Architecture Impact Continued Pre-training for African Languagescited by 12
  25. 2024Sabi'a-2: A New Generation of Portuguese Large Language Modelscited by 10
  26. 2024SGHateCheck: Functional Tests for Detecting Hate Speech in Low-Resource Languages of Singaporecited by 10
  27. 2024Facilitating large language model Russian adaptation with Learned Embedding Propagationcited by 8
  28. 2025SeaLLMs-Audio: Large Audio-Language Models for Southeast Asiacited by 8
  29. 2024The Power of Question Translation Training in Multilingual Reasoning: Broadened Scope and Deepened Insightscited by 7
  30. 2025Building High-Quality Datasets for Portuguese LLMs: From Common Crawl Snapshots to Industrial-Grade Corporacited by 7
  31. 2024Training Bilingual LMs with Data Constraints in the Targeted Languagecited by 6
  32. 2024Adapting Chat Language Models Using Only Target Unlabeled Language Datacited by 6
  33. 2025CAReDiO: Cultural Alignment via Representativeness and Distinctiveness Guided Data Optimizationcited by 6
  34. 2026Stop Taking Tokenizers for Granted: They Are Core Design Decisions in Large Language Modelscited by 6
  35. 2024Improving Language Models Trained on Translated Data with Continual Pre-Training and Dictionary Learning Analysiscited by 5
  36. 2025Cultural Alignment in Large Language Models Using Soft Prompt Tuningcited by 5
  37. 2024Cracking the Code: Multi-domain LLM Evaluation on Real-World Professional Exams in Indonesiacited by 4
  38. 2024KAHANI: Culturally-Nuanced Visual Storytelling Tool for Non-Western Culturescited by 4

50 most cited of 104 All 104 (JSON)