Are Multilingual Language Models an Off-ramp for Under-resourced Languages? Will we arrive at Digital Language Equality in Europe in 2030?
Not on the map. Why?
Abstract (arXiv)
Large language models (LLMs) demonstrate unprecedented capabilities and define the state of the art for almost all natural language processing (NLP) tasks and also for essentially all Language Technology (LT) applications. LLMs can only be trained for languages for which a sufficient amount of pre-training data is available, effectively excluding many languages that are typically characterised as under-resourced. However, there is both circumstantial and empirical evidence that multilingual LLMs, which have been trained using data sets that cover multiple languages (including under-resourced ones), do exhibit strong capabilities for some of these under-resourced languages. Eventually, this approach may have the potential to be a technological off-ramp for those under-resourced languages for which "native" LLMs, and LLM-based technologies, cannot be developed due to a lack of training data. This paper, which concentrates on European languages, examines this idea, analyses the current situation in terms of technology support and summarises related work. The article concludes by focusing on the key open questions that need to be answered for the approach to be put into practice in a systematic way.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 14 of its 32 references that are in the map. First posted on arXiv 18 Feb 2025.
Builds on
14 of 32 references.
References · 14
- 2020Measuring Massive Multitask Language Understandingcited by 7,963
- 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
- 2023MEGA: Multilingual Evaluation of Generative AIcited by 293
- 2024Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluationcited by 191
- 2023MADLAD-400: A Multilingual And Document-Level Large Audited Datasetcited by 181
- 2023CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languagescited by 174
- 2024EuroLLM: Multilingual Language Models for Europecited by 106
- 2024LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuningcited by 89
- 2024LLMs Are Few-Shot In-Context Low-Resource Language Learnerscited by 68
- 2024A New Massive Multilingual Dataset for High-Performance Language Technologiescited by 56
- 2024Towards Multilingual LLM Evaluation for European Languagescited by 30
- 2024Teuken-7B-Base & Teuken-7B-Instruct: Towards European LLMscited by 22
2 more
- 2023Not all layers are equally as important: Every Layer Counts BERTcited by 21
- 2024What Drives Performance in Multilingual Language Models?cited by 15
Built on by
Citing papers per year: 2026: 1.
1 paper.