# MERaLiON-TextLLM: Cross-Lingual Understanding of Large Language Models in Chinese, Indonesian, Malay, and Singlish

arXiv: https://arxiv.org/abs/2501.08335 · HTML: https://mlsanity.com/paper/2501.08335/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 8 references that are in the map.
- First posted on arXiv 21 Dec 2024.

## Builds on (most cited first, 5 of 5)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Models](https://mlsanity.com/paper/2305.08322/) (2023)
- [SeaEval for Multilingual Foundation Models: From Cross-Lingual Alignment to Cultural Reasoning](https://mlsanity.com/paper/2309.04766/) (2023)
- [Large Language Models Only Pass Primary School Exams in Indonesia: A Comprehensive Test on IndoMMLU](https://mlsanity.com/paper/2310.04928/) (2023)

## Built on by (most cited first, 4 of 4)

- [The State of Multilingual LLM Safety Research: From Measuring the Language Gap to Mitigating It](https://mlsanity.com/paper/2505.24119/) (2025)
- [Batayan: A Filipino NLP benchmark for evaluating Large Language Models](https://mlsanity.com/paper/2502.14911/) (2025)
- [BURMESE-SAN: Burmese NLP Benchmark for Evaluating Large Language Models](https://mlsanity.com/paper/2602.18788/) (2026)
- [Words at Play: Benchmarking Audio Pun Understanding in Large Audio-Language Models](https://mlsanity.com/paper/2603.18678/) (2026)

## Abstract (arXiv)

Multilingual large language models (MLLMs) have shown impressive capabilities across a variety of languages. However, efficacy can differ greatly between different language families, especially for those with limited linguistic resources. This report presents MERaLiON-TextLLM, a series of open-source language models specifically tailored to improve understanding and generation in Chinese, Indonesian, Malay, and Singlish. The initial released model is built on Llama-3-8B-Base and refined through a meticulously crafted process of continued pre-training and weight merging. Our approach achieves performance improvements across benchmarks in these languages, exceeding the capabilities of the official Llama-3 models. We provide the model checkpoints as a resource to support further research and development in cross-lingual language understanding.