# SeaLLMs 3: Open Foundation and Chat Multilingual Large Language Models for Southeast Asian Languages

arXiv: https://arxiv.org/abs/2407.19672 · HTML: https://mlsanity.com/paper/2407.19672/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 38 references that are in the map.
- First posted on arXiv 29 Jul 2024.

## Builds on (most cited first, 20 of 20)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [Gemma: Open Models Based on Gemini Research and Technology](https://mlsanity.com/paper/2403.08295/) (2024)
- [TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages](https://mlsanity.com/paper/2003.05002/) (2020)
- [Measuring and Improving Consistency in Pretrained Language Models](https://mlsanity.com/paper/2102.01017/) (2021)
- [MEGA: Multilingual Evaluation of Generative AI](https://mlsanity.com/paper/2303.12528/) (2023)
- [MADLAD-400: A Multilingual And Document-Level Large Audited Dataset](https://mlsanity.com/paper/2309.04662/) (2023)
- [CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languages](https://mlsanity.com/paper/2309.09400/) (2023)
- [How do Large Language Models Handle Multilingualism?](https://mlsanity.com/paper/2402.18815/) (2024)
- [Aya 23: Open Weight Releases to Further Multilingual Progress](https://mlsanity.com/paper/2405.15032/) (2024)
- [SeaLLMs -- Large Language Models for Southeast Asia](https://mlsanity.com/paper/2312.00738/) (2023)
- [LLaMA Beyond English: An Empirical Study on Language Capability Transfer](https://mlsanity.com/paper/2401.01055/) (2024)
- [One Country, 700+ Languages: NLP Challenges for Underrepresented Languages and Dialects in Indonesia](https://mlsanity.com/paper/2203.13357/) (2022)
- [Democratizing LLMs for Low-Resource Languages by Leveraging their English Dominant Abilities with Linguistically-Diverse Prompts](https://mlsanity.com/paper/2306.11372/) (2023)

## Built on by (most cited first, 30 of 36)

- [SEA-LION: Southeast Asian Languages in One Network](https://mlsanity.com/paper/2504.05747/) (2025)
- [Scaling Language-Centric Omnimodal Representation Learning](https://mlsanity.com/paper/2510.11693/) (2025)
- [SEA-HELM: Southeast Asian Holistic Evaluation of Language Models](https://mlsanity.com/paper/2502.14301/) (2025)
- [Sailor2: Sailing in South-East Asia with Inclusive Multilingual LLMs](https://mlsanity.com/paper/2502.12982/) (2025)
- [When Less Language is More: Language-Reasoning Disentanglement Makes LLMs Better Multilingual Reasoners](https://mlsanity.com/paper/2505.15257/) (2025)
- [Disentangling Language and Culture for Evaluating Multilingual Large Language Models](https://mlsanity.com/paper/2505.24635/) (2025)
- [RSafe: Incentivizing proactive reasoning to build robust and adaptive LLM safeguards](https://mlsanity.com/paper/2506.07736/) (2025)
- [SeaExam and SeaBench: Benchmarking LLMs with Local Multilingual Questions in Southeast Asia](https://mlsanity.com/paper/2502.06298/) (2025)
- [Typhoon 2: A Family of Open Text and Multimodal Thai Large Language Models](https://mlsanity.com/paper/2412.13702/) (2024)
- [Understanding Multilingualism in Mixture-of-Experts LLMs: Routing Mechanism, Expert Specialization, and Layerwise Steering](https://mlsanity.com/paper/2601.14050/) (2026)
- [From RAG to Agentic RAG for Faithful Islamic Question Answering](https://mlsanity.com/paper/2601.07528/) (2026)
- [SeaLLMs-Audio: Large Audio-Language Models for Southeast Asia](https://mlsanity.com/paper/2511.01670/) (2025)
- [Analyzing LLMs' Knowledge Boundary Cognition Across Languages Through the Lens of Internal Representations](https://mlsanity.com/paper/2504.13816/) (2025)
- [Batayan: A Filipino NLP benchmark for evaluating Large Language Models](https://mlsanity.com/paper/2502.14911/) (2025)
- [Document Intelligence in the Era of Large Language Models: A Survey](https://mlsanity.com/paper/2510.13366/) (2025)
- [Language of Thought Shapes Output Diversity in Large Language Models](https://mlsanity.com/paper/2601.11227/) (2026)
- [Low-rank Optimization Trajectories Modeling for LLM RLVR Acceleration](https://mlsanity.com/paper/2604.11446/) (2026)
- [Unveiling the Influence of Amplifying Language-Specific Neurons](https://mlsanity.com/paper/2507.22581/) (2025)
- [FilBench: Can LLMs Understand and Generate Filipino?](https://mlsanity.com/paper/2508.03523/) (2025)
- [OpenSeal: Good, Fast, and Cheap Construction of an Open-Source Southeast Asian LLM via Parallel Data](https://mlsanity.com/paper/2602.02266/) (2026)
- [Representing the Under-Represented: Cultural and Core Capability Benchmarks for Developing Thai Large Language Models](https://mlsanity.com/paper/2410.04795/) (2024)
- [SEALGuard: Safeguarding the Multilingual Conversations in Southeast Asian Languages for LLM Software Systems](https://mlsanity.com/paper/2507.08898/) (2025)
- [Toxicity Red-Teaming: Benchmarking LLM Safety in Singapore's Low-Resource Languages](https://mlsanity.com/paper/2509.15260/) (2025)
- [Extracting and Combining Abilities For Building Multi-lingual Ability-enhanced Large Language Models](https://mlsanity.com/paper/2410.07825/) (2024)
- [CAMEO: Collection of Multilingual Emotional Speech Corpora](https://mlsanity.com/paper/2505.11051/) (2025)
- [Comprehensive Evaluation on Lexical Normalization: Boundary-Aware Approaches for Unsegmented Languages](https://mlsanity.com/paper/2505.22273/) (2025)
- [Pruning General Large Language Models into Customized Expert Models](https://mlsanity.com/paper/2506.02561/) (2025)
- [Multimodal Evaluation of Russian-language Architectures](https://mlsanity.com/paper/2511.15552/) (2025)
- [Unlocking Cognitive Capabilities and Analyzing the Perception-Logic Trade-off](https://mlsanity.com/paper/2602.23730/) (2026)
- [VietJobs: A Vietnamese Job Advertisement Dataset](https://mlsanity.com/paper/2603.05262/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have shown remarkable abilities across various tasks, yet their development has predominantly centered on high-resource languages like English and Chinese, leaving low-resource languages underserved. To address this disparity, we present SeaLLMs 3, the latest iteration of the SeaLLMs model family, tailored for Southeast Asian languages. This region, characterized by its rich linguistic diversity, has lacked adequate language technology support. SeaLLMs 3 aims to bridge this gap by covering a comprehensive range of languages spoken in this region, including English, Chinese, Indonesian, Vietnamese, Thai, Tagalog, Malay, Burmese, Khmer, Lao, Tamil, and Javanese. Leveraging efficient language enhancement techniques and a specially constructed instruction tuning dataset, SeaLLMs 3 significantly reduces training costs while maintaining high performance and versatility. Our model excels in tasks such as world knowledge, mathematical reasoning, translation, and instruction following, achieving state-of-the-art performance among similarly sized models. Additionally, we prioritized safety and reliability by addressing both general and culture-specific considerations and incorporated mechanisms to reduce hallucinations. This work underscores the importance of inclusive AI, showing that advanced LLM capabilities can benefit underserved linguistic and cultural communities.