mlsanityData to 7 Oct 2026

Opportunities and Challenges of Large Language Models for Low-Resource Languages in Humanities Research

arXiv 2412.04497 ↗First posted 30 Nov 2024cs.CL cs.AIcited by 26

Not on the map. Why?

Abstract (arXiv)

Low-resource languages serve as invaluable repositories of human history, embodying cultural evolution and intellectual diversity. Despite their significance, these languages face critical challenges, including data scarcity and technological limitations, which hinder their comprehensive study and preservation. Recent advancements in large language models (LLMs) offer transformative opportunities for addressing these challenges, enabling innovative methodologies in linguistic, historical, and cultural research. This study systematically evaluates the applications of LLMs in low-resource language research, encompassing linguistic variation, historical documentation, cultural expressions, and literary analysis. By analyzing technical frameworks, current methodologies, and ethical considerations, this paper identifies key challenges such as data accessibility, model adaptability, and cultural sensitivity. Given the cultural, historical, and linguistic richness inherent in low-resource languages, this work emphasizes interdisciplinary collaboration and the development of customized models as promising avenues for advancing research in this domain. By underscoring the potential of integrating artificial intelligence with the humanities to preserve and study humanity's linguistic and cultural heritage, this study fosters global efforts towards safeguarding intellectual diversity.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 80 of its 152 references that are in the map. First posted on arXiv 30 Nov 2024.

Builds on

80 of 152 references.

References · 80

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2024The Llama 3 Herd of Modelscited by 14,192
  5. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  6. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  7. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  8. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  9. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  10. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  11. 2017A Survey on Multi-Task Learningcited by 838
  12. 2019How multilingual is Multilingual BERT?cited by 642
38 more
  1. 2022Mixture-of-Experts with Expert Choice Routingcited by 611
  2. 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 443
  3. 2024TrustLLM: Trustworthiness in Large Language Modelscited by 280
  4. 2019Massively Multilingual Neural Machine Translation in the Wild: Findings and Challengescited by 253
  5. 2016Diachronic Word Embeddings Reveal Statistical Laws of Semantic Changecited by 219
  6. 2023BiomedGPT: A Generalist Vision-Language Foundation Model for Diverse Biomedical Taskscited by 201
  7. 2016Transfer Learning for Low-Resource Neural Machine Translationcited by 188
  8. 2020A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarioscited by 170
  9. 2023Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Modelscited by 152
  10. 2023We're Afraid Language Models Aren't Modeling Ambiguitycited by 124
  11. 2024Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Modelscited by 124
  12. 2021Neural Machine Translation for Low-Resource Languages: A Surveycited by 108
  13. 2020SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detectioncited by 105
  14. 2020GLUECoS : An Evaluation Benchmark for Code-Switched NLPcited by 101
  15. 2022The Geometry of Multilingual Language Model Representationscited by 94
  16. 2020Analysing Lexical Semantic Change with Contextualised Word Representationscited by 92
  17. 2020Low-resource Languages: A Review of Past Work and Future Challengescited by 90
  18. 2024Revolutionizing Finance with LLMs: An Overview of Applications and Insightscited by 88
  19. 2023Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedbackcited by 86
  20. 2024Understanding LLMs: A Comprehensive Overview from Training to Inferencecited by 58
  21. 2023An Iterative Optimizing Framework for Radiology Report Summarization with ChatGPTcited by 57
  22. 2024A Comprehensive Review of Multimodal Large Language Models: Performance and Challenges Across Different Taskscited by 57
  23. 2020Handwritten Optical Character Recognition (OCR): A Comprehensive Systematic Literature Review (SLR)cited by 56
  24. 2023When Brain-inspired AI Meets AGIcited by 54
  25. 2021AmericasNLI: Evaluating Zero-shot Natural Language Understanding of Pretrained Multilingual Models in Truly Low-resource Languagescited by 50
  26. 2024CultureLLM: Incorporating Cultural Differences into Large Language Modelscited by 49
  27. 2024DIALECTBENCH: A NLP Benchmark for Dialects, Varieties, and Closely-Related Languagescited by 49
  28. 2020Masakhane -- Machine Translation For Africacited by 48
  29. 2024AraDiCE: Benchmarks for Dialectal and Cultural Capabilities in LLMscited by 46
  30. 2023Radiology-Llama2: Best-in-Class Large Language Model for Radiologycited by 42
  31. 2023AD-AutoGPT: An Autonomous GPT for Alzheimer's Disease Infodemiologycited by 34
  32. 2023Context Matters: A Strategy to Pre-train Language Model for Science Educationcited by 32
  33. 2020It's not Greek to mBERT: Inducing Word-Level Translations from Multilingual BERTcited by 30
  34. 2022Discovering Language-neutral Sub-networks in Multilingual Language Modelscited by 28
  35. 2023Overcoming Catastrophic Forgetting in Massively Multilingual Continual Learningcited by 27
  36. 2023Differentiate ChatGPT-generated and Human-written Medical Textscited by 26
  37. 2024CulturePark: Boosting Cross-cultural Understanding in Large Language Modelscited by 24
  38. 2024Large Language Models for Manufacturingcited by 24

50 most cited of 80 All 80 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 16; 2026: 10.

26 papers.

Sort:

By area:

  1. 2025Between Circuits and Chomsky: Pre-pretraining on Formal Languages Imparts Linguistic Biasescited by 36
  2. 2025Beyond Log Likelihood: Probability-Based Objectives for Supervised Fine-Tuning across the Model Capability Continuumcited by 16
  3. 2025Relic: Enhancing Reward Model Generalization for Low-Resource Indic Languages with Few-Shot Examplescited by 6
  4. 2025Whispering in Amharic: Fine-tuning Whisper for Low-resource Languagecited by 5
  5. 2025NusaAksara: A Multimodal and Multilingual Benchmark for Preserving Indonesian Indigenous Scriptscited by 4
  6. 2025Large Language Models for Bioinformaticscited by 3
  7. 2025Enhancing Low-Resource Minority Language Translation with LLMs and Retrieval-Augmented Generation for Cultural Nuancescited by 2
  8. 2025Bridging Gaps in Hate Speech Detection: Meta-Collections and Benchmarks for Low-Resource Iberian Languagescited by 2
  9. 2025Is Lying Only Sinful in Islam? Exploring Religious Bias in Multilingual Large Language Models Across Major Religionscited by 1
  10. 2025Think Outside the Data: Colonial Biases and Systemic Issues in Automated Moderation Pipelines for Low-Resource Languagescited by 0
  11. 2025Bridging Technology and Humanities: Evaluating the Impact of Large Language Models on Social Sciences Research with DeepSeek-R1cited by 0
  12. 2025Advancing Uto-Aztecan Language Technologies: A Case Study on the Endangered Comanche Languagecited by 0
14 more
  1. 2025Mafoko: Structuring and Building Open Multilingual Terminologies for South African NLPcited by 0
  2. 2025AdaDocVQA: Adaptive Framework for Long Document Visual Question Answering in Low-Resource Settingscited by 0
  3. 2025Limited Reference, Reliable Generation: A Two-Component Framework for Tabular Data Generation in Low-Data Regimescited by 0
  4. 2025Do Large Language Models Grasp The Grammar? Evidence from Grammar-Book-Guided Probing in Luxembourgishcited by 0
  5. 2026AfroScope: A Framework for Studying the Linguistic Landscape of Africacited by 0
  6. 2026ECHO: An Open Research Platform for Evaluation of Chat, Human Behavior, and Outcomescited by 0
  7. 2026CGRA-DeBERTa Concept Guided Residual Augmentation Transformer for Theologically Islamic Understandingcited by 0
  8. 2026LLM Probe: Evaluating LLMs for Low-Resource Languagescited by 0
  9. 2026Cross-Lingual Transfer and Parameter-Efficient Adaptation in the Turkic Language Family: A Theoretical Framework for Low-Resource Language Modelscited by 0
  10. 2026In Data or Invisible: Toward a Better Digital Representation of Low-Resource Languages with Knowledge Graphscited by 0
  11. 2026PersLitEval: Fine-grained Benchmark and Evaluation of LLMs on Persian Literature Questionscited by 0
  12. 2026Simorgh at SemEval-2026 task 7: Region-Aware Hybrid Retrieval for Low-Resource Cultural Reasoning in Multilingual Question Answeringcited by 0
  13. 2026Benchmarking Open-Weight Foundation Models for Global AI Technical Governancecited by 0
  14. 2026MoLGE: Mixture of Language Group Experts for Efficient Scaling of Massively Multilingual Speech Recognitioncited by 0