mlsanityData to 7 Oct 2026

MiLiC-Eval: Benchmarking Multilingual LLMs for China's Minority Languages

arXiv 2503.01150 ↗First posted 3 Mar 2025cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) excel in high-resource languages but struggle with low-resource languages (LRLs), particularly those spoken by minority communities in China, such as Tibetan, Uyghur, Kazakh, and Mongolian. To systematically track the progress in these languages, we introduce MiLiC-Eval, a benchmark designed for minority languages in China, featuring 24K instances across 9 tasks. MiLiC-Eval focuses on underrepresented writing systems. Its parallelism between tasks and languages can provide a faithful and fine-grained assessment of linguistic and problem-solving skills. Our evaluation reveals that open-source LLMs perform poorly on syntax-intensive tasks and multi-script languages. We further demonstrate how MiLiC-Eval can help advance LRL research in handling diverse writing systems and understanding the process of language adaptation.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 29 of its 46 references that are in the map. First posted on arXiv 3 Mar 2025.

Builds on

29 of 46 references.

References · 29

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2024Qwen2 Technical Reportcited by 2,076
  4. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  5. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 738
  6. 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 591
  7. 2024Understanding the planning of LLM agents: A surveycited by 414
  8. 2021XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languagescited by 346
  9. 2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 300
  10. 2023MEGA: Multilingual Evaluation of Generative AIcited by 291
  11. 2024Tower: An Open Multilingual Large Language Model for Translation-Related Taskscited by 205
  12. 2023Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Modelscited by 186
17 more
  1. 2023SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialectscited by 149
  2. 2024Aya Expanse: Combining Research Breakthroughs for a New Multilingual Frontiercited by 121
  3. 2023A Confederacy of Models: a Comprehensive Evaluation of LLMs on Creative Writingcited by 98
  4. 2023SeaEval for Multilingual Foundation Models: From Cross-Lingual Alignment to Cultural Reasoningcited by 94
  5. 2022BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Promptingcited by 90
  6. 2024Understanding and Mitigating Language Confusion in LLMscited by 75
  7. 2023BUFFET: Benchmarking Large Language Models for Few-shot Cross-lingual Transfercited by 68
  8. 2024IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languagescited by 55
  9. 2023FinGPT: Large Generative Models for a Small Languagecited by 49
  10. 2023WanJuan: A Comprehensive Multimodal Dataset for Advancing English and Chinese Large Modelscited by 48
  11. 2024LLaMAX: Scaling Linguistic Horizons of LLM by Enhancing Translation Capabilities Beyond 100 Languagescited by 47
  12. 2023XTREME-UP: A User-Centric Scarce-Data Benchmark for Under-Represented Languagescited by 34
  13. 2024Teaching Large Language Models an Unseen Language on the Flycited by 29
  14. 2024P-MMEval: A Parallel Multilingual Multitask Benchmark for Consistent Evaluation of LLMscited by 29
  15. 2023Taxi1500: A Multilingual Dataset for Text Classification in 1500 Languagescited by 23
  16. 2024Mitigating the Language Mismatch and Repetition Issues in LLM-based Machine Translation via Model Editingcited by 18
  17. 2024BayLing 2: A Multilingual Large Language Model with Efficient Language Alignmentcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025TFD: A Comprehensive Structured Tibetan Foundation Dataset for Low-Resource Language Processing and Large-Scale Modelingcited by 3
  2. 2025LaoBench: A Large-Scale Multidimensional Lao Benchmark for Large Language Modelscited by 2
  3. 2025Do Large Language Models Grasp The Grammar? Evidence from Grammar-Book-Guided Probing in Luxembourgishcited by 0
  4. 2026Efficient Low-Resource Language Adaptation via Multi-Source Dynamic Logit Fusioncited by 0