mlsanityData to 7 Oct 2026

LLMs for Extremely Low-Resource Finno-Ugric Languages

arXiv 2410.18902 ↗First posted 24 Oct 2024cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

The advancement of large language models (LLMs) has predominantly focused on high-resource languages, leaving low-resource languages, such as those in the Finno-Ugric family, significantly underrepresented. This paper addresses this gap by focusing on V~oro, Livonian, and Komi. We cover almost the entire cycle of LLM creation, from data collection to instruction tuning and evaluation. Our contributions include developing multilingual base and instruction-tuned models; creating evaluation benchmarks, including the smugri-MT-bench multi-turn conversational benchmark; and conducting human evaluation. We intend for this work to promote linguistic diversity, ensuring that lesser-resourced languages can benefit from advancements in NLP.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 41 of its 60 references that are in the map. First posted on arXiv 24 Oct 2024.

Builds on

41 of 60 references.

References · 41

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  5. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  6. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  7. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 789
  8. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  9. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 712
  10. 2023Prometheus: Inducing Fine-grained Evaluation Capability in Language Modelscited by 558
  11. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 556
  12. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 518
29 more
  1. 2024Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Modelscited by 431
  2. 2023How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resourcescited by 405
  3. 2023Scaling Data-Constrained Language Modelscited by 391
  4. 2023Efficient and Effective Text Encoding for Chinese LLaMA and Alpacacited by 302
  5. 2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 300
  6. 2023MEGA: Multilingual Evaluation of Generative AIcited by 291
  7. 2023Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedbackcited by 189
  8. 2023MADLAD-400: A Multilingual And Document-Level Large Audited Datasetcited by 180
  9. 2024Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuningcited by 174
  10. 2023CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languagescited by 172
  11. 2023SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialectscited by 149
  12. 2023UniMax: Fairer and more Effective Language Sampling for Large-Scale Multilingual Pretrainingcited by 147
  13. 2024LLaMA Beyond English: An Empirical Study on Language Capability Transfercited by 94
  14. 2023Do Multilingual Language Models Think Better in English?cited by 87
  15. 2023Are Large Language Model-based Evaluators the Solution to Scaling Up Multilingual Evaluation?cited by 87
  16. 2023Extrapolating Large Language Models to Non-English by Aligning Languagescited by 69
  17. 2023Breaking Language Barriers in Multilingual Mathematical Reasoning: Insights and Observationscited by 58
  18. 2023MEGAVERSE: Benchmarking Large Language Models Across Languages, Modalities, Models and Taskscited by 57
  19. 2023FinGPT: Large Generative Models for a Small Languagecited by 49
  20. 2024Latxa: An Open Language Model and Evaluation Suite for Basquecited by 42
  21. 2024Enhancing Multilingual Capabilities of Large Language Models through Self-Distillation from Resource-Rich Languagescited by 36
  22. 2023LLaMAntino: LLaMA 2 Models for Effective Text Generation in Italian Languagecited by 24
  23. 2024Teaching Llama a New Language Through Cross-Lingual Knowledge Transfercited by 23
  24. 2024Poro 34B and the Blessing of Multilingualitycited by 18
  25. 2020EstBERT: A Pretrained Language-Specific BERT for Estoniancited by 16
  26. 2024Komodo: A Linguistic Expedition into Indonesia's Regional Languagescited by 9
  27. 2021Extremely low-resource machine translation for closely related languagescited by 7
  28. 2024Rethinking LLM Language Adaptation: A Case Study on Chinese Mixtralcited by 6
  29. 2024Self-Translate-Train: Enhancing Cross-Lingual Transfer of Large Language Models via Inherent Capabilitycited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Instructing Large Language Models for Low-Resource Languages: A Systematic Study for Basquecited by 12
  2. 2026EstLLM: Enhancing Estonian Capabilities in Multilingual LLMs via Continued Pretraining and Post-Trainingcited by 1
  3. 2025Exploring NLP Benchmarks in an Extremely Low-Resource Settingcited by 0