mlsanityData to 7 Oct 2026

Sailor2: Sailing in South-East Asia with Inclusive Multilingual LLMs

arXiv 2502.12982 ↗First posted 18 Feb 2025cs.CL cs.AI cs.LGcited by 17

Not on the map. Why?

Abstract (arXiv)

Sailor2 is a family of cutting-edge multilingual language models for South-East Asian (SEA) languages, available in 1B, 8B, and 20B sizes to suit diverse applications. Building on Qwen2.5, Sailor2 undergoes continuous pre-training on 500B tokens (400B SEA-specific and 100B replay tokens) to support 13 SEA languages while retaining proficiency in Chinese and English. Sailor2-20B model achieves a 50-50 win rate against GPT-4o across SEA languages. We also deliver a comprehensive cookbook on how to develop the multilingual model in an efficient manner, including five key aspects: data curation, pre-training, post-training, model customization and evaluation. We hope that Sailor2 model (Apache 2.0 license) will drive language development in the SEA region, and Sailor2 cookbook will inspire researchers to build more inclusive LLMs for other under-served languages.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 74 of its 88 references that are in the map. First posted on arXiv 18 Feb 2025.

Builds on

74 of 88 references.

References · 74

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  5. 2017Deep reinforcement learning from human preferencescited by 3,846
  6. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  7. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  8. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  9. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  10. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 988
  11. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 951
  12. 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 906
38 more
  1. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 825
  2. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 694
  3. 2021Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LMcited by 621
  4. 2024OLMo: Accelerating the Science of Language Modelscited by 599
  5. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 552
  6. 2021ByT5: Towards a token-free future with pre-trained byte-to-byte modelscited by 504
  7. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 486
  8. 2023Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruningcited by 408
  9. 2024DataComp-LM: In search of the next generation of training sets for language modelscited by 381
  10. 2020XCOPA: A Multilingual Dataset for Causal Commonsense Reasoningcited by 330
  11. 2024Skywork-Reward: Bag of Tricks for Reward Modeling in LLMscited by 322
  12. 2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 300
  13. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 292
  14. 2024Direct Language Model Alignment from Online AI Feedbackcited by 257
  15. 2024WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wildcited by 211
  16. 2024Many-Shot In-Context Learningcited by 203
  17. 2024Disentangling Length from Quality in Direct Preference Optimizationcited by 194
  18. 2024Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluationcited by 181
  19. 2024Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuningcited by 174
  20. 2024RegMix: Data Mixture as Regression for Language Model Pre-trainingcited by 172
  21. 2024BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languagescited by 171
  22. 2024OpenCoder: The Open Cookbook for Top-Tier Code Large Language Modelscited by 127
  23. 2024Leave No Document Behind: Benchmarking Long-Context LLMs with Extended Multi-Doc QAcited by 121
  24. 2024Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabulariescited by 113
  25. 2023M3Exam: A Multilingual, Multimodal, Multilevel Benchmark for Examining Large Language Modelscited by 109
  26. 2023SeaLLMs -- Large Language Models for Southeast Asiacited by 104
  27. 2023LLM360: Towards Fully Transparent Open-Source LLMscited by 100
  28. 2024LLaMA Pro: Progressive LLaMA with Block Expansioncited by 99
  29. 2023SeaEval for Multilingual Foundation Models: From Cross-Lingual Alignment to Cultural Reasoningcited by 94
  30. 2024GliDe with a CaPE: A Low-Hassle Method to Accelerate Speculative Decodingcited by 74
  31. 2023Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Modelscited by 68
  32. 2024MAP-Neo: Highly Capable and Transparent Bilingual Large Language Model Seriescited by 67
  33. 2022Language Modelling with Pixelscited by 53
  34. 2023Large Language Models Only Pass Primary School Exams in Indonesia: A Comprehensive Test on IndoMMLUcited by 53
  35. 2024Bootstrapping Language Models with DPO Implicit Rewardscited by 53
  36. 2023FinGPT: Large Generative Models for a Small Languagecited by 49
  37. 2023Improving Language Plasticity via Pretraining with Active Forgettingcited by 43
  38. 2024SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languagescited by 41

50 most cited of 74 All 74 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 9; 2026: 8.

17 papers.

Sort:

By area:

  1. 2025When Less Language is More: Language-Reasoning Disentanglement Makes LLMs Better Multilingual Reasonerscited by 16
  2. 2025MuBench: Assessment of Multilingual Capabilities of Large Language Models Across 61 Languagescited by 13
  3. 2025Global PIQA: Evaluating Commonsense Reasoning Across 100+ Languages and Culturescited by 8
  4. 2025SeaLLMs-Audio: Large Audio-Language Models for Southeast Asiacited by 7
  5. 2025Batayan: A Filipino NLP benchmark for evaluating Large Language Modelscited by 5
  6. 2025Compass-V2 Technical Reportcited by 2
  7. 2026Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcyclingcited by 2
  8. 2025VietBinoculars: A Zero-Shot Approach for Detecting Vietnamese LLM-Generated Textcited by 1
  9. 2025LLaMAX2: Your Translation-Enhanced Model also Performs Well in Reasoningcited by 1
  10. 2026BURMESE-SAN: Burmese NLP Benchmark for Evaluating Large Language Modelscited by 1
  11. 2026VietJobs: A Vietnamese Job Advertisement Datasetcited by 1
  12. 2026XL-SafetyBench: A Country-Grounded Cross-Cultural Benchmark for LLM Safety and Cultural Sensitivitycited by 1
5 more
  1. 2026BYOL: Bring Your Own Language Into LLMscited by 0
  2. 2025Orchestrating Heterogeneous Experts: A Scalable MoE Framework with Anisotropy-Preserving Fusioncited by 0
  3. 2026Phoenix-VL 1.5 Medium Technical Reportcited by 0
  4. 2026A Comparative Study of Language Models for Khmer Retrieval-Augmented Question Answeringcited by 0
  5. 2026In-Place Tokenizer Expansion for Pre-trained LLMscited by 0