mlsanityData to 7 Oct 2026

Latxa: An Open Language Model and Evaluation Suite for Basque

arXiv 2403.20266 ↗First posted 29 Mar 2024cs.CL cs.AI cs.LGcited by 42

Not on the map. Why?

Abstract (arXiv)

We introduce Latxa, a family of large language models for Basque ranging from 7 to 70 billion parameters. Latxa is based on Llama 2, which we continue pretraining on a new Basque corpus comprising 4.3M documents and 4.2B tokens. Addressing the scarcity of high-quality benchmarks for Basque, we further introduce 4 multiple choice evaluation datasets: EusProficiency, comprising 5,169 questions from official language proficiency exams; EusReading, comprising 352 reading comprehension questions; EusTrivia, comprising 1,715 trivia questions from 5 knowledge areas; and EusExams, comprising 16,774 questions from public examinations. In our extensive evaluation, Latxa outperforms all previous open models we compare to by a large margin. In addition, it is competitive with GPT-4 Turbo in language proficiency and understanding, despite lagging behind in reading comprehension and knowledge-intensive tasks. Both the Latxa family of models, as well as our new pretraining corpora and evaluation datasets, are publicly available under open licenses. Our suite enables reproducible research on methods to build LLMs for low-resource languages.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Builds on 29 of its 50 references that are in the map. First posted on arXiv 29 Mar 2024.

Builds on

29 of 50 references.

References · 29

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2020Scaling Laws for Neural Language Modelscited by 6,411
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  7. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  8. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  9. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  10. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  11. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  12. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
17 more
  1. 2024Yi: Open Foundation Models by 01.AIcited by 706
  2. 2024OLMo: Accelerating the Science of Language Modelscited by 599
  3. 2024Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Researchcited by 475
  4. 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 339
  5. 2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 300
  6. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 292
  7. 2021Few-shot Learning with Multilingual Language Modelscited by 278
  8. 2023MADLAD-400: A Multilingual And Document-Level Large Audited Datasetcited by 180
  9. 2023CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languagescited by 172
  10. 2022mGPT: Few-Shot Learners Go Multilingualcited by 141
  11. 2023LLM360: Towards Fully Transparent Open-Source LLMscited by 100
  12. 2020Translation Artifacts in Cross-lingual Transfer Learningcited by 95
  13. 2022BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Promptingcited by 90
  14. 2024CroissantLLM: A Truly Bilingual French-English Language Modelcited by 52
  15. 2023FinGPT: Large Generative Models for a Small Languagecited by 49
  16. 2020Give your Text Representation Models some Love: the Case for Basquecited by 28
  17. 2023A Comparative Study of Open-Source Large Language Models, GPT-4 and Claude 2: Multiple-Choice Test Taking in Nephrologycited by 26

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 10; 2025: 23; 2026: 9.

42 papers.

Sort:

By area:

  1. 2024INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledgecited by 78
  2. 2025Instructing Large Language Models for Low-Resource Languages: A Systematic Study for Basquecited by 12
  3. 2026Replaying pre-training data improves fine-tuningcited by 12
  4. 2024LL"aMmlein: Transparent, Compact and Competitive German-Only Language Models from Scratchcited by 9
  5. 2025ScholaWrite: A Dataset of End-to-End Scholarly Writing Processcited by 8
  6. 2025HKCanto-Eval: A Benchmark for Evaluating Cantonese Language Understanding and Cultural Comprehension in LLMscited by 8
  7. 2025Colombian Waitresses y Jueces canadienses: Gender and Country Biases in Occupation Recommendations from LLMscited by 7
  8. 2025The Harmonic Structure of Information Contourscited by 7
  9. 2025Truth Knows No Language: Evaluating Truthfulness Beyond Englishcited by 5
  10. 2025IberBench: LLM Evaluation on Iberian Languagescited by 5
  11. 2025Emergent Abilities of Large Language Models under Continued Pretraining for Language Adaptationcited by 5
  12. 2025A Practitioner's Guide to Building ASR Models for Low-Resource Languages: A Case Study on Scottish Gaeliccited by 5
30 more
  1. 2024Pipeline Analysis for Developing Instruct LLMs in Low-Resource Languages: A Case Study on Basquecited by 4
  2. 2024Why We Build Local Large Language Models: An Observational Analysis from 35 Japanese and Multilingual LLMscited by 4
  3. 2025Whisper-LM: Improving ASR Models with Language Models for Low-Resource Languagescited by 4
  4. 2024XNLIeu: a dataset for cross-lingual NLI in Basquecited by 3
  5. 2024Automating Easy Read Text Segmentationcited by 3
  6. 2024Open Generative Large Language Models for Galiciancited by 3
  7. 2024LLMs for Extremely Low-Resource Finno-Ugric Languagescited by 3
  8. 2025In-context Learning vs. Instruction Tuning: The Case of Small and Multilingual Language Modelscited by 3
  9. 2025Grow Up and Merge: Scaling Strategies for Efficient Language Adaptationcited by 3
  10. 2026Life After Benchmark Saturation: A Case Study of CORE-Benchcited by 3
  11. 2024The #Somos600M Project: Generating NLP resources that represent the diversity of the languages from LATAM, the Caribbean, and Spaincited by 2
  12. 2025Lost in Variation? Evaluating NLI Performance in Basque and Spanish Geographical Variantscited by 2
  13. 2025La Leaderboard: A Large Language Model Leaderboard for Spanish Varieties and Languages of Spain and Latin Americacited by 2
  14. 2025Mangosteen: An Open Thai Corpus for Language Model Pretrainingcited by 2
  15. 2025Estonian Native Large Language Model Benchmarkcited by 2
  16. 2025Low-Resource Dialect Adaptation of Large Language Models: A French Dialect Case-Studycited by 2
  17. 2024What's New in My Data? Novelty Exploration via Contrastive Generationcited by 1
  18. 2025UrduLLaMA 1.0: Dataset Curation, Preprocessing, and Evaluation in Low-Resource Settingscited by 1
  19. 2025Evaluating Compact LLMs for Zero-Shot Iberian Language Tasks on End-User Devicescited by 1
  20. 2025Qomhra: A Bilingual Irish and English Large Language Modelcited by 1
  21. 2026EstLLM: Enhancing Estonian Capabilities in Multilingual LLMs via Continued Pretraining and Post-Trainingcited by 1
  22. 2026GaelEval: Benchmarking LLM Performance for Scottish Gaeliccited by 1
  23. 2025Euskarazko lehen C1 ebaluatzaile automatikoacited by 0
  24. 2025BERnaT: Basque Encoders for Representing Natural Textual Diversitycited by 0
  25. 2025Automatic Essay Scoring and Feedback Generation in Basque Language Learningcited by 0
  26. 2026A Catalog of Basque Dialectal Resources: Online Collections and Standard-to-Dialectal Adaptationscited by 0
  27. 2026Merge and Conquer: Instructing Multilingual Models by Adding Target Language Weightscited by 0
  28. 2026MameLoshnLM: Yiddish Language Model and Evaluation Benchmarkcited by 0
  29. 2026Reading the News: Adapting Large Language Models to Swedish Journalism Through Continued Pre-Trainingcited by 0
  30. 2026From Zero to Hero: An Open LLM Ecosystem for Armeniancited by 0