Multilinguality at the Edge: Developing Language Models for the Global South
Not on the map. Why?
Abstract (arXiv)
Where and how language models (LMs) are deployed determines who can benefit from them. However, there are several challenges that prevent effective deployment of LMs in non-English-speaking and hardware constrained communities in the Global South. We call this challenge the last mile: the intersection of multilinguality and edge deployment, where the goals are aligned but the technical requirements often compete. Studying these two fields together is both a need, as linguistically diverse communities often face the most severe infrastructure constraints, and an opportunity, as edge and multilingual NLP research remain largely siloed. To understand the state of the art and the challenges of combining the two areas, we survey 232 papers that tackle this problem across the language modelling pipeline, from data collection to development and deployment. We also discuss open questions and provide actionable recommendations for different stakeholders in the NLP ecosystem. Finally, we hope that this work contributes to the development of inclusive and equitable language technologies.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 95 of its 165 references that are in the map. First posted on arXiv 23 Apr 2026.
Builds on
95 of 165 references.
References · 95
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2020Scaling Laws for Neural Language Modelscited by 6,411
- 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
- 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
- 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
- 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,414
- 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
- 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 840
- 2023TIES-Merging: Resolving Interference When Merging Modelscited by 795
- 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 789
- 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 443
38 more
- 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compressioncited by 443
- 2024LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compressioncited by 343
- 2023Efficient and Effective Text Encoding for Chinese LLaMA and Alpacacited by 302
- 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 292
- 2025Small Language Models are the Future of Agentic AIcited by 254
- 2021Quality at a Glance: An Audit of Web-Crawled Multilingual Datasetscited by 241
- 2024MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Casescited by 216
- 2023Language Model Tokenizers Introduce Unfairness Between Languagescited by 207
- 2023Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Modelscited by 186
- 2023Power Hungry Processing: Watts Driving the Cost of AI Deployment?cited by 186
- 2023A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicitycited by 180
- 2024Towards Measuring and Modeling "Culture" in LLMs: A Surveycited by 166
- 2022Lifting the Curse of Multilinguality by Pre-training Modular Transformerscited by 145
- 2023Pretraining task diversity and the emergence of non-Bayesian in-context learning for regressioncited by 138
- 2025FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Languagecited by 134
- 2024Aya Expanse: Combining Research Breakthroughs for a New Multilingual Frontiercited by 121
- 2020UNKs Everywhere: Adapting Multilingual Language Models to New Scriptscited by 113
- 2022Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipecited by 108
- 2023Tokenizer Choice For LLM Training: Negligible or Crucial?cited by 105
- 2024A Review on Edge Large Language Models: Design, Execution, and Applicationscited by 98
- 2024AgentInstruct: Toward Generative Teaching with Agentic Flowscited by 88
- 2022Efficient Methods for Natural Language Processing: A Surveycited by 78
- 2025How Hungry is AI? Benchmarking Energy, Water, and Carbon Footprint of LLM Inferencecited by 73
- 2024A Survey on Large Language Models for Critical Societal Domains: Finance, Healthcare, and Lawcited by 72
- 2024TinyAgent: Function Calling at the Edgecited by 55
- 2024Learning to Compress Prompt in Natural Language Formatscited by 54
- 2023Chat Vector: A Simple Approach to Equip LLMs with Instruction Following and Model Alignment in New Languagescited by 52
- 2024M-RewardBench: Evaluating Reward Models in Multilingual Settingscited by 50
- 2021A Primer on Pretrained Multilingual Language Modelscited by 49
- 2021The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translationcited by 46
- 2024From Tokens to Words: On the Inner Lexicon of LLMscited by 46
- 2023AfroBench: How Good are Large Language Models on African Languages?cited by 43
- 2024Natural Language Processing for the Legal Domain: A Survey of Tasks, Datasets, Models, and Challengescited by 42
- 2025The State of Multilingual LLM Safety Research: From Measuring the Language Gap to Mitigating Itcited by 42
- 2024How Does Quantization Affect Multilingual LLMs?cited by 41
- 2025Universal Cross-Tokenizer Distillation via Approximate Likelihood Matchingcited by 40
- 2025M-Prometheus: A Suite of Open Multilingual LLM Judgescited by 37
- 2024Breaking the Curse of Multilinguality with Cross-lingual Expert Language Modelscited by 36
50 most cited of 95 All 95 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.