mlsanityData to 7 Oct 2026

EuroLLM: Multilingual Language Models for Europe

arXiv 2409.16235 ↗First posted 24 Sep 2024cs.CLcited by 103

Not on the map. Why?

Abstract (arXiv)

The quality of open-weight LLMs has seen significant improvement, yet they remain predominantly focused on English. In this paper, we introduce the EuroLLM project, aimed at developing a suite of open-weight multilingual LLMs capable of understanding and generating text in all official European Union languages, as well as several additional relevant languages. We outline the progress made to date, detailing our data collection and filtering process, the development of scaling laws, the creation of our multilingual tokenizer, and the data mix and modeling configurations. Additionally, we release our initial models: EuroLLM-1.7B and EuroLLM-1.7B-Instruct and report their performance on multilingual general benchmarks and machine translation.

Key facts. Cited by 103 papers in the map (data through 7 Oct 2026). Builds on 32 of its 46 references that are in the map. First posted on arXiv 24 Sep 2024.

Builds on

32 of 46 references.

References · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2020Scaling Laws for Neural Language Modelscited by 6,411
  5. 2016Layer Normalizationcited by 5,579
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  7. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  8. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  9. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  10. 2023Mistral 7Bcited by 3,065
  11. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  12. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
20 more
  1. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  2. 2024Mixtral of Expertscited by 1,785
  3. 2020GLU Variants Improve Transformercited by 1,724
  4. 2019Root Mean Square Layer Normalizationcited by 1,425
  5. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  6. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  7. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
  8. 2020On Layer Normalization in the Transformer Architecturecited by 906
  9. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 694
  10. 2024TinyLlama: An Open-Source Small Language Modelcited by 604
  11. 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 544
  12. 2022The Stack: 3 TB of permissively licensed source codecited by 354
  13. 2023Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedbackcited by 189
  14. 2023CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languagescited by 172
  15. 2019On the Use of ArXiv as a Datasetcited by 101
  16. 2022What Language Model to Train if You Have One Million GPU Hours?cited by 92
  17. 2018A Walk with SGDcited by 74
  18. 2023Scaling Laws for Multilingual Neural Machine Translationcited by 27
  19. 2024Preliminary WMT24 Ranking of General MT Systems and LLMscited by 20
  20. 2024Recovering document annotations for sentence-level bitextcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 63; 2026: 34.

103 papers.

Sort:

By area:

  1. 2025LEXam: Benchmarking Legal Reasoning on 340 Law Examscited by 52
  2. 2025mmBERT: A Modern Multilingual Encoder with Annealed Language Learningcited by 39
  3. 2025BOUQuET: dataset, Benchmark and Open initiative for Universal Quality Evaluation in Translationcited by 21
  4. 2025Granary: Speech Recognition and Translation Dataset in 25 European Languagescited by 21
  5. 2024Teuken-7B-Base & Teuken-7B-Instruct: Towards European LLMscited by 20
  6. 2025Localized Cultural Knowledge is Conserved and Controllable in Large Language Modelscited by 19
  7. 2025Multilingual Language Model Pretraining using Machine-translated Datacited by 17
  8. 2025Krikri: Advancing Open Large Language Models for Greekcited by 13
  9. 2025The Landscape of Arabic Large Language Models (ALLMs): A New Era for Arabic Language Technologycited by 12
  10. 2025Instructing Large Language Models for Low-Resource Languages: A Systematic Study for Basquecited by 12
  11. 2025Multilingual Contextualization of Large Language Models for Document-Level Machine Translationcited by 11
  12. 2025Beyond Text Compression: Evaluating Tokenizers Across Scalescited by 11
38 more
  1. 2024LL"aMmlein: Transparent, Compact and Competitive German-Only Language Models from Scratchcited by 9
  2. 2025TurBLiMP: A Turkish Benchmark of Linguistic Minimal Pairscited by 9
  3. 2025Tokenization Standards for Linguistic Integrity: Turkish as a Benchmarkcited by 8
  4. 2025Gaperon: A Peppered English-French Generative Language Model Suitecited by 8
  5. 2025From TOWER to SPIRE: Adding the Speech Modality to a Translation-Specialist LLMcited by 7
  6. 2025A quantitative analysis of semantic information in deep representations of text and imagescited by 7
  7. 2025Simultaneous Translation with Offline Speech and LLM Models in CUNI Submission to IWSLT 2025cited by 7
  8. 2025Speech LLMs in Low-Resource Scenarios: Data Volume Requirements and the Impact of Pretraining on High-Resource Languagescited by 7
  9. 2025Tokens with Meaning: A Hybrid Tokenization Approach for Turkishcited by 7
  10. 2025Minimal Pair-Based Evaluation of Code-Switchingcited by 6
  11. 2025Open-sci-ref-0.01: open and reproducible reference baselines for language model and dataset comparisoncited by 6
  12. 2025Extending Automatic Machine Translation Evaluation to Book-Length Documentscited by 6
  13. 2025Bielik 11B v3: Multilingual Large Language Model for European Languagescited by 6
  14. 2026GreekMMLU: A Native-Sourced Multitask Benchmark for Evaluating Language Models in Greekcited by 6
  15. 2024Context-Aware or Context-Insensitive? Assessing LLMs' Performance in Document-Level Translationcited by 5
  16. 2024From English-Centric to Effective Bilingual: LLMs with Custom Tokenizers for Underrepresented Languagescited by 5
  17. 2024Multilingual Pretraining Using a Large Corpus Machine-Translated from a Single Source Languagecited by 5
  18. 2025How Much Do LLMs Hallucinate across Languages? On Realistic Multilingual Estimation of LLM Hallucinationcited by 5
  19. 2025Assumed Identities: Quantifying Gender Bias in Machine Translation of Gender-Ambiguous Occupational Termscited by 5
  20. 2025Pensez: Less Data, Better Reasoning -- Rethinking French LLMcited by 5
  21. 2025IberBench: LLM Evaluation on Iberian Languagescited by 5
  22. 2025UNCERTAINTY-LINE: Length-Invariant Estimation of Uncertainty for Large Language Modelscited by 5
  23. 2025NAVER LABS Europe Submission to the Instruction-following Trackcited by 5
  24. 2025Training-free LLM Merging for Multi-task Learningcited by 5
  25. 2026When Meanings Meet: Investigating the Emergence and Quality of Shared Concept Spaces during Multilingual Language Model Trainingcited by 5
  26. 2025Translate Smart, not Hard: Cascaded Translation Systems with Quality-Aware Deferralcited by 4
  27. 2025EuroGEST: Investigating gender stereotypes in multilingual language modelscited by 4
  28. 2025The Eloquence team submission for task 1 of MLC-SLM challengecited by 4
  29. 2025CUS-QA: Local-Knowledge-Oriented Open-Ended Question Answering Datasetcited by 4
  30. 2025Dougal Dil .Icslemede Tokenizasyon Standartlari ve "Olcc"um"u: T"urkcce "Uzerinden B"uy"uk Dil Modellerinin Karcsilacstirmali Analizicited by 4
  31. 2025In-context Learning vs. Instruction Tuning: The Case of Small and Multilingual Language Modelscited by 3
  32. 2025Got Compute, but No Data: Lessons From Post-training a Finnish LLMcited by 3
  33. 2025ExPLAIND: Unifying Model, Data, and Training Attribution to Study Model Behaviorcited by 3
  34. 2025OWL: Probing Cross-Lingual Recall of Memorized Texts via World Literaturecited by 3
  35. 2025Luth: Efficient French Specialization for Small Language Models and Cross-Lingual Transfercited by 3
  36. 2026On the Credibility of Evaluating LLMs using Survey Questionscited by 3
  37. 2026BERT-as-a-Judge: A Robust Alternative to Lexical Methods for Efficient Reference-Based LLM Evaluationcited by 3
  38. 2025Testing Low-Resource Language Support in LLMs Using Language Proficiency Exams: the Case of Luxembourgishcited by 2

50 most cited of 103 All 103 (JSON)