# Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontiers

arXiv: https://arxiv.org/abs/2404.04925 · HTML: https://mlsanity.com/paper/2404.04925/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 262 of its 352 references that are in the map.
- First posted on arXiv 7 Apr 2024.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 30 of 262)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [PaLM-E: An Embodied Multimodal Language Model](https://mlsanity.com/paper/2303.03378/) (2023)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding](https://mlsanity.com/paper/2006.16668/) (2020)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [COMET: A Neural Framework for MT Evaluation](https://mlsanity.com/paper/2009.09025/) (2020)
- [Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions](https://mlsanity.com/paper/2212.10509/) (2022)
- [XNLI: Evaluating Cross-lingual Sentence Representations](https://mlsanity.com/paper/1809.05053/) (2018)
- [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://mlsanity.com/paper/2302.04023/) (2023)
- [Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks](https://mlsanity.com/paper/2204.07705/) (2022)
- [PaLI: A Jointly-Scaled Multilingual Language-Image Model](https://mlsanity.com/paper/2209.06794/) (2022)
- [XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization](https://mlsanity.com/paper/2003.11080/) (2020)
- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [OpenAssistant Conversations -- Democratizing Large Language Model Alignment](https://mlsanity.com/paper/2304.07327/) (2023)
- [AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models](https://mlsanity.com/paper/2304.06364/) (2023)
- [Baichuan 2: Open Large-scale Language Models](https://mlsanity.com/paper/2309.10305/) (2023)
- [WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct](https://mlsanity.com/paper/2308.09583/) (2023)
- [On the Cross-lingual Transferability of Monolingual Representations](https://mlsanity.com/paper/1910.11856/) (2019)

## Built on by (most cited first, 30 of 51)

- [Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models](https://mlsanity.com/paper/2503.09567/) (2025)
- [A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment](https://mlsanity.com/paper/2504.15585/) (2025)
- [Large Language Models Meet NLP: A Survey](https://mlsanity.com/paper/2405.12819/) (2024)
- [Unlocking the Capabilities of Thought: A Reasoning Boundary Framework to Quantify and Optimize Chain-of-Thought](https://mlsanity.com/paper/2410.05695/) (2024)
- [What Makes and Breaks Safety Fine-tuning? A Mechanistic Study](https://mlsanity.com/paper/2407.10264/) (2024)
- [The State of Multilingual LLM Safety Research: From Measuring the Language Gap to Mitigating It](https://mlsanity.com/paper/2505.24119/) (2025)
- [The Hidden Dimensions of LLM Alignment: A Multi-Dimensional Analysis of Orthogonal Safety Directions](https://mlsanity.com/paper/2502.09674/) (2025)
- [Language Ranker: A Metric for Quantifying LLM Performance Across High and Low-Resource Languages](https://mlsanity.com/paper/2404.11553/) (2024)
- [100 Days After DeepSeek-R1: A Survey on Replication Studies and More Directions for Reasoning Language Models](https://mlsanity.com/paper/2505.00551/) (2025)
- [Undesirable Memorization in Large Language Models: A Survey](https://mlsanity.com/paper/2410.02650/) (2024)
- [Language Imbalance Driven Rewarding for Multilingual Self-improving](https://mlsanity.com/paper/2410.08964/) (2024)
- [Scaling Laws for Multilingual Language Models](https://mlsanity.com/paper/2410.12883/) (2024)
- [Large Language Models Lack Understanding of Character Composition of Words](https://mlsanity.com/paper/2405.11357/) (2024)
- [Multilingual Blending: LLM Safety Alignment Evaluation with Language Mixture](https://mlsanity.com/paper/2407.07342/) (2024)
- [Youtu-GraphRAG: Vertically Unified Agents for Graph Retrieval-Augmented Complex Reasoning](https://mlsanity.com/paper/2508.19855/) (2025)
- [AutoCAP: Towards Automatic Cross-lingual Alignment Planning for Zero-shot Chain-of-Thought](https://mlsanity.com/paper/2406.13940/) (2024)
- [Multilingual Large Language Models: A Systematic Survey](https://mlsanity.com/paper/2411.11072/) (2024)
- [SLAM: Towards Efficient Multilingual Reasoning via Selective Language Alignment](https://mlsanity.com/paper/2501.03681/) (2025)
- [RLSF: Fine-tuning LLMs via Symbolic Feedback](https://mlsanity.com/paper/2405.16661/) (2024)
- [XIFBench: Evaluating Large Language Models on Multilingual Instruction Following](https://mlsanity.com/paper/2503.07539/) (2025)
- [Improving Multilingual Language Models by Aligning Representations through Steering](https://mlsanity.com/paper/2505.12584/) (2025)
- [Crosslingual On-Policy Self-Distillation for Multilingual Reasoning](https://mlsanity.com/paper/2605.09548/) (2026)
- [MrGuard: A Multilingual Reasoning Guardrail for Universal LLM Safety](https://mlsanity.com/paper/2504.15241/) (2025)
- [Structure-aware Domain Knowledge Injection for Large Language Models](https://mlsanity.com/paper/2407.16724/) (2024)
- [Lens: Rethinking Multilingual Enhancement for Large Language Models](https://mlsanity.com/paper/2410.04407/) (2024)
- [Multilingual Prompt Engineering in Large Language Models: A Survey Across NLP Tasks](https://mlsanity.com/paper/2505.11665/) (2025)
- [Transfer Attack for Bad and Good: Explain and Boost Adversarial Transferability across Multimodal Large Language Models](https://mlsanity.com/paper/2405.20090/) (2024)
- [Foundation Models for Low-Resource Language Education (Vision Paper)](https://mlsanity.com/paper/2412.04774/) (2024)
- [Reasoning Transfer for an Extremely Low-Resource and Endangered Language: Bridging Languages Through Sample-Efficient Language Understanding](https://mlsanity.com/paper/2504.02890/) (2025)
- [Mixed Text Recognition with Efficient Parameter Fine-Tuning and Transformer](https://mlsanity.com/paper/2404.12734/) (2024)

## Abstract (arXiv)

Multilingual Large Language Models are capable of using powerful Large Language Models to handle and respond to queries in multiple languages, which achieves remarkable success in multilingual natural language processing tasks. Despite these breakthroughs, there still remains a lack of a comprehensive survey to summarize existing approaches and recent developments in this field. To this end, in this paper, we present a thorough review and provide a unified perspective to summarize the recent progress as well as emerging trends in multilingual large language models (MLLMs) literature. The contributions of this paper can be summarized: (1) First survey: to our knowledge, we take the first step and present a thorough review in MLLMs research field according to multi-lingual alignment; (2) New taxonomy: we offer a new and unified perspective to summarize the current progress of MLLMs; (3) New frontiers: we highlight several emerging frontiers and discuss the corresponding challenges; (4) Abundant resources: we collect abundant open-source resources, including relevant papers, data corpora, and leaderboards. We hope our work can provide the community with quick access and spur breakthrough research in MLLMs.