mlsanityData to 14 Oct 2026

EuroLLM-22B: Technical Report

arXiv 2602.05879 ↗First posted 5 Feb 2026cs.CL cs.AI cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

This report presents EuroLLM-22B, a large language model trained from scratch to support the needs of European citizens by covering all 24 official European Union languages and 11 additional languages. EuroLLM addresses the issue of European languages being underrepresented and underserved in existing open large language models. We provide a comprehensive overview of EuroLLM-22B's development, including tokenizer design, architectural specifications, data filtering, and training procedures. Across a broad set of multilingual benchmarks, EuroLLM-22B demonstrates strong performance in reasoning, instruction following, and translation, achieving results competitive with models of comparable size. To support future research, we release our base and instruction-tuned models, our multilingual web pretraining data and updated EuroBlocks instruction datasets, as well as our pre-training and evaluation codebases.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 57 of its 83 references that are in the map. First posted on arXiv 5 Feb 2026.

Builds on

57 of 83 references.

References · 57

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  6. 2025Qwen3 Technical Reportcited by 8,820
  7. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  8. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  9. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  10. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  11. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  12. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
38 more
  1. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  2. 2023Let's Verify Step by Stepcited by 3,789
  3. 2023Mistral 7Bcited by 3,164
  4. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  5. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  6. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  7. 2025Gemma 3 Technical Reportcited by 1,824
  8. 2024Mixtral of Expertscited by 1,823
  9. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  10. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  11. 2020GLU Variants Improve Transformercited by 1,804
  12. 2019Root Mean Square Layer Normalizationcited by 1,480
  13. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  14. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,254
  15. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  16. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
  17. 2020On Layer Normalization in the Transformer Architecturecited by 922
  18. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 735
  19. 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 548
  20. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 530
  21. 2022The Stack: 3 TB of permissively licensed source codecited by 359
  22. 2024Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothingcited by 346
  23. 2024Skywork-Reward: Bag of Tricks for Reward Modeling in LLMscited by 327
  24. 2023Effective Long-Context Scaling of Foundation Modelscited by 303
  25. 2020Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translationcited by 277
  26. 2024HelpSteer2: Open-source dataset for training top-performing reward modelscited by 236
  27. 2019WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipediacited by 223
  28. 2024Tower: An Open Multilingual Large Language Model for Translation-Related Taskscited by 206
  29. 2024OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Datacited by 200
  30. 2023Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedbackcited by 190
  31. 2019CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEBcited by 177
  32. 2023CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languagescited by 174
  33. 2024Nemotron-CC: Transforming Common Crawl into a Refined Long-Horizon Pretraining Datasetcited by 160
  34. 2019CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairscited by 136
  35. 2024Liger Kernel: Efficient Triton Kernels for LLM Trainingcited by 130
  36. 2024Aya Expanse: Combining Research Breakthroughs for a New Multilingual Frontiercited by 129
  37. 2019On the Use of ArXiv as a Datasetcited by 103
  38. 2025MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluationcited by 79

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 9.

9 papers.

  1. 2026BERT-as-a-Judge: A Robust Alternative to Lexical Methods for Efficient Reference-Based LLM Evaluationcited by 4
  2. 2026Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcyclingcited by 3
  3. 2026A Sovereign, Open-Source Foundation Model for German and Englishcited by 1
  4. 2026Improving Term Evaluation in Machine Translation: Variation Matterscited by 1
  5. 2026Maistros: A Greek Large Language Model Adapted Through Knowledge Distillation From Large Reasoning Modelscited by 0
  6. 2026Fully Open Meditron: An Auditable Pipeline for Clinical LLMscited by 0
  7. 2026How Far Can Machine Translation Quality Take You? Extrinsic Discourse Evaluation in Goal-Oriented Setupscited by 0
  8. 2026CantoneseLLM v2: Reasoning in a Low-Resource Languagecited by 0
  9. 2026Project Greenhouse: Progress Toward Fully Open and Sovereign Agentic Searchcited by 0