# IndoLEM and IndoBERT: A Benchmark Dataset and Pre-trained Language Model for Indonesian NLP

arXiv: https://arxiv.org/abs/2011.00677 · HTML: https://mlsanity.com/paper/2011.00677/

## Key facts

- Cited by 58 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Nov 2020.

## Built on by (most cited first, 30 of 58)

- [Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model](https://mlsanity.com/paper/2402.07827/) (2024)
- [AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processing](https://mlsanity.com/paper/2108.05542/) (2021)
- [Bactrian-X: Multilingual Replicable Instruction-Following Models with Low-Rank Adaptation](https://mlsanity.com/paper/2305.15011/) (2023)
- [NusaX: Multilingual Parallel Sentiment Dataset for 10 Indonesian Local Languages](https://mlsanity.com/paper/2205.15960/) (2022)
- [One Country, 700+ Languages: NLP Challenges for Underrepresented Languages and Dialects in Indonesia](https://mlsanity.com/paper/2203.13357/) (2022)
- [IndoNLG: Benchmark and Resources for Evaluating Indonesian Natural Language Generation](https://mlsanity.com/paper/2104.08200/) (2021)
- [Large Language Models Only Pass Primary School Exams in Indonesia: A Comprehensive Test on IndoMMLU](https://mlsanity.com/paper/2310.04928/) (2023)
- [Discourse Probing of Pretrained Language Models](https://mlsanity.com/paper/2104.05882/) (2021)
- [NusaCrowd: Open Source Initiative for Indonesian NLP Resources](https://mlsanity.com/paper/2212.09648/) (2022)
- [KinyaBERT: a Morphology-aware Kinyarwanda Language Model](https://mlsanity.com/paper/2203.08459/) (2022)
- [IndoBERTweet: A Pretrained Language Model for Indonesian Twitter with Effective Domain-Specific Vocabulary Initialization](https://mlsanity.com/paper/2109.04607/) (2021)
- [IndoNLI: A Natural Language Inference Dataset for Indonesian](https://mlsanity.com/paper/2110.14566/) (2021)
- [GreenPLM: Cross-Lingual Transfer of Monolingual Pre-Trained Language Models at Almost No Cost](https://mlsanity.com/paper/2211.06993/) (2022)
- [InstructAlign: High-and-Low Resource Language Alignment via Continual Crosslingual Instruction Tuning](https://mlsanity.com/paper/2305.13627/) (2023)
- [Cendol: Open Instruction-tuned Generative Large Language Models for Indonesian Languages](https://mlsanity.com/paper/2404.06138/) (2024)
- [ScandEval: A Benchmark for Scandinavian Natural Language Processing](https://mlsanity.com/paper/2304.00906/) (2023)
- [QASiNa: Religious Domain Question Answering using Sirah Nabawiyah](https://mlsanity.com/paper/2310.08102/) (2023)
- [Evaluating the Efficacy of Summarization Evaluation across Languages](https://mlsanity.com/paper/2106.01478/) (2021)
- [This is the way: designing and compiling LEPISZCZE, a comprehensive NLP benchmark for Polish](https://mlsanity.com/paper/2211.13112/) (2022)
- [CINO: A Chinese Minority Pre-trained Language Model](https://mlsanity.com/paper/2202.13558/) (2022)
- [Bridging the Digital Divide: Performance Variation across Socio-Economic Factors in Vision-Language Models](https://mlsanity.com/paper/2311.05746/) (2023)
- [Llama-3-Nanda-10B-Chat: An Open Generative Large Language Model for Hindi](https://mlsanity.com/paper/2504.06011/) (2025)
- [NLEBench+NorGLM: A Comprehensive Empirical Analysis and Benchmark Dataset for Generative Language Models in Norwegian](https://mlsanity.com/paper/2312.01314/) (2023)
- [Crowdsource, Crawl, or Generate? Creating SEA-VL, a Multicultural Vision-Language Dataset for Southeast Asia](https://mlsanity.com/paper/2503.07920/) (2025)
- [A Comprehensive Analysis of Adapter Efficiency](https://mlsanity.com/paper/2305.07491/) (2023)
- [The State of Large Language Models for African Languages: Progress and Challenges](https://mlsanity.com/paper/2506.02280/) (2025)
- [Zero-shot Sentiment Analysis in Low-Resource Languages Using a Multilingual Sentiment Lexicon](https://mlsanity.com/paper/2402.02113/) (2024)
- [Which Student is Best? A Comprehensive Knowledge Distillation Exam for Task-Specific BERT Models](https://mlsanity.com/paper/2201.00558/) (2022)
- [GeoGLUE: A GeoGraphic Language Understanding Evaluation Benchmark](https://mlsanity.com/paper/2305.06545/) (2023)
- [NusaWrites: Constructing High-Quality Corpora for Underrepresented and Extremely Low-Resource Languages](https://mlsanity.com/paper/2309.10661/) (2023)

## Abstract (arXiv)

Although the Indonesian language is spoken by almost 200 million people and the 10th most spoken language in the world, it is under-represented in NLP research. Previous work on Indonesian has been hampered by a lack of annotated datasets, a sparsity of language resources, and a lack of resource standardization. In this work, we release the IndoLEM dataset comprising seven tasks for the Indonesian language, spanning morpho-syntax, semantics, and discourse. We additionally release IndoBERT, a new pre-trained language model for Indonesian, and evaluate it over IndoLEM, in addition to benchmarking it against existing resources. Our experiments show that IndoBERT achieves state-of-the-art performance over most of the tasks in IndoLEM.