mlsanityData to 14 Oct 2026

The State of Large Language Models for African Languages: Progress and Challenges

arXiv 2506.02280 ↗First posted 2 Jun 2025cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) are transforming Natural Language Processing (NLP), but their benefits are largely absent for Africa's 2,000 low-resource languages. This paper comparatively analyzes African language coverage across six LLMs, eight Small Language Models (SLMs), and six Specialized SLMs (SSLMs). The evaluation covers language coverage, training sets, technical limitations, script problems, and language modelling roadmaps. The work identifies 42 supported African languages and 23 available public data sets, and it shows a big gap where four languages (Amharic, Swahili, Afrikaans, and Malagasy) are always treated while there is over 98% of unsupported African languages. Moreover, the review shows that just Latin, Arabic, and Ge'ez scripts are identified while 20 active scripts are neglected. Some of the primary challenges are lack of data, tokenization biases, computational costs being very high, and evaluation issues. These issues demand language standardization, corpus development by the community, and effective adaptation methods for African languages.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 34 of its 41 references that are in the map. First posted on arXiv 2 Jun 2025.

Builds on

34 of 41 references.

References · 34

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  6. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  7. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  8. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  9. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  10. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  11. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  12. 2019Cross-lingual Language Model Pretrainingcited by 1,090
22 more
  1. 2020Language-agnostic BERT Sentence Embeddingcited by 627
  2. 2021ByT5: Towards a token-free future with pre-trained byte-to-byte modelscited by 509
  3. 2020AraBERT: Transformer-based Model for Arabic Language Understandingcited by 362
  4. 2020How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Modelscited by 318
  5. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 295
  6. 2021CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representationcited by 232
  7. 2020Are All Languages Created Equal in Multilingual BERT?cited by 207
  8. 2021MasakhaNER: Named Entity Recognition for African Languagescited by 155
  9. 2022Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuningcited by 149
  10. 2023Hallucinations in Large Multilingual Translation Modelscited by 129
  11. 2024A Comprehensive Survey of Small Language Models in the Era of Large Language Models: Techniques, Enhancements, Applications, Collaboration with LLMs, and Trustworthinesscited by 127
  12. 2023Glot500: Scaling Multilingual Corpora and Language Models to 500 Languagescited by 116
  13. 2024The Language Barrier: Dissecting Safety Challenges of LLMs in Multilingual Contextscited by 94
  14. 2021AraT5: Text-to-Text Transformers for Arabic Language Generationcited by 63
  15. 2020IndoLEM and IndoBERT: A Benchmark Dataset and Pre-trained Language Model for Indonesian NLPcited by 58
  16. 2020AraGPT2: Pre-Trained Transformer for Arabic Language Generationcited by 55
  17. 2023FinGPT: Large Generative Models for a Small Languagecited by 49
  18. 2022AfroLM: A Self-Active Learning-based Multilingual Pretrained Language Model for 23 African Languagescited by 46
  19. 2021IndicBART: A Pre-trained Model for Indic Natural Language Generationcited by 39
  20. 2024Scaling Laws for Fact Memorization of Large Language Modelscited by 33
  21. 2024EthioLLM: Multilingual Large Language Models for Ethiopian Languages with Task Evaluationcited by 8
  22. 2024EthioMT: Parallel Corpus for Low-resource Ethiopian Languagescited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 4.

8 papers.

  1. 2026Where Are We At with Automatic Speech Recognition for the Bambara Language?cited by 1
  2. 2025Automatic Correction of Writing Anomalies in Hausa Textscited by 0
  3. 2025The Rise of AfricaNLP: A Survey of Contributions, Contributors, Community Impact, and Bibliometric Analysiscited by 0
  4. 2025AfriStereo: A Culturally Grounded Dataset for Evaluating Stereotypical Bias in Large Language Modelscited by 0
  5. 2025Opportunities and Challenges of Natural Language Processing for Low-Resource Senegalese Languages in Social Science Researchcited by 0
  6. 2026AfroScope: A Framework for Studying the Linguistic Landscape of Africacited by 0
  7. 2026Sample-Size Scaling of the African Languages NLI Evaluationcited by 0
  8. 2026Are Language Models Script-Aware?cited by 0