mlsanityData to 13 Oct 2026

How Good are Commercial Large Language Models on African Languages?

arXiv 2305.06530 ↗First posted 11 May 2023cs.CL cs.AI cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

Recent advancements in Natural Language Processing (NLP) has led to the proliferation of large pretrained language models. These models have been shown to yield good performance, using in-context learning, even on unseen tasks and languages. They have also been exposed as commercial APIs as a form of language-model-as-a-service, with great adoption. However, their performance on African languages is largely unknown. We present a preliminary analysis of commercial large language models on two tasks (machine translation and text classification) across eight African languages, spanning different language families and geographical areas. Our results suggest that commercial language models produce below-par performance on African languages. We also find that they perform better on text classification than machine translation. In general, our findings present a call-to-action to ensure African languages are well represented in commercial large language models, given their growing popularity.

Key facts. Cited by 8 papers in the map (data through 13 Oct 2026). Builds on 46 of its 62 references that are in the map. First posted on arXiv 11 May 2023.

Builds on

46 of 62 references.

References · 46

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  2. 2020Language Models are Few-Shot Learnerscited by 32,320
  3. 2022Training language models to follow instructions with human feedbackcited by 15,856
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,549
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,322
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,376
  7. 2020Scaling Laws for Neural Language Modelscited by 6,494
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,765
  9. 2022Scaling Instruction-Finetuned Language Modelscited by 3,007
  10. 2022Emergent Abilities of Large Language Modelscited by 2,499
  11. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,379
  12. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,793
34 more
  1. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,468
  2. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,302
  3. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,191
  4. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 841
  5. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 792
  6. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 668
  7. 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 544
  8. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 525
  9. 2020XCOPA: A Multilingual Dataset for Causal Commonsense Reasoningcited by 332
  10. 2022Black-Box Tuning for Language-Model-as-a-Servicecited by 256
  11. 2023Prompting Large Language Model for Machine Translation: A Case Studycited by 249
  12. 2017What do Neural Machine Translation Models Learn about Morphology?cited by 166
  13. 2020Participatory Research for Low-resourced Machine Translation: A Case Study in African Languagescited by 159
  14. 2020Tangled up in BLEU: Reevaluating the Evaluation of Automatic Machine Translation Evaluation Metricscited by 156
  15. 2021MasakhaNER: Named Entity Recognition for African Languagescited by 155
  16. 2022Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuningcited by 149
  17. 2023The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Datasetcited by 148
  18. 2021Systematic Inequalities in Language Technology Performance across the World's Languagescited by 119
  19. 2020BLEU might be Guilty but References are not Innocentcited by 110
  20. 2022A Few Thousand Translations Go a Long Way! Leveraging Pre-trained Models for African News Translationcited by 97
  21. 2020Translation Artifacts in Cross-lingual Transfer Learningcited by 96
  22. 2021Language Models are Few-shot Multilingual Learnerscited by 92
  23. 2022Language Contamination Helps Explain the Cross-lingual Capabilities of English Pretrained Modelscited by 92
  24. 2020Transfer Learning and Distant Supervision for Multilingual Transformer Models: A Study on African Languagescited by 52
  25. 2021Discrete and Soft Prompting for Multilingual Modelscited by 49
  26. 2022AfroLM: A Self-Active Learning-based Multilingual Pretrained Language Model for 23 African Languagescited by 46
  27. 2022SERENGETI: Massively Multilingual Language Models for Africacited by 40
  28. 2022What a Creole Wants, What a Creole Needscited by 27
  29. 2020It's Easier to Translate out of English than into it: Measuring Neural Translation Difficulty by Cross-Mutual Informationcited by 26
  30. 2022Beyond Counting Datasets: A Survey of Multilingual Dataset Construction and Necessary Resourcescited by 25
  31. 2021On Language Models for Creolescited by 17
  32. 2019PidginUNMT: Unsupervised Neural Machine Translation from West African Pidgin to Englishcited by 11
  33. 2020Unsupervised Pidgin Text Generation By Pivoting English Data and Self-Trainingcited by 9
  34. 2020Towards Supervised and Unsupervised Neural Machine Translation Baselines for Nigerian Pidgincited by 8

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2024: 2; 2025: 5; 2026: 1.

8 papers.

  1. 2024BayLing 2: A Multilingual Large Language Model with Efficient Language Alignmentcited by 8
  2. 2025Lugha-Llama: Adapting Large Language Models for African Languagescited by 8
  3. 2024Language Modelling Approaches to Adaptive Machine Translationcited by 3
  4. 2025Cross-Cultural Fashion Design via Interactive Large Language Models and Diffusion Modelscited by 0
  5. 2025Weak Supervision Dynamic KL-Weighted Diffusion Models Guided by Large Language Modelscited by 0
  6. 2025Multi-granular Training Strategies for Robust Multi-hop Reasoning Over Noisy and Heterogeneous Knowledge Sourcescited by 0
  7. 2025Context-Enhanced Granular Edit Representation for Efficient and Accurate ASR Post-editingcited by 0
  8. 2026Large Language Models for Low-Resource Languages: A Conceptual Framework for an Electronic Explanatory Dictionary of the Tajik Languagecited by 0