mlsanityData to 14 Oct 2026

SERENGETI: Massively Multilingual Language Models for Africa

arXiv 2212.10785 ↗First posted 21 Dec 2022cs.CL cs.AIcited by 40

Not on the map. Why?

Abstract (arXiv)

Multilingual pretrained language models (mPLMs) acquire valuable, generalizable linguistic information during pretraining and have advanced the state of the art on task-specific finetuning. To date, only ~31 out of ~2,000 African languages are covered in existing language models. We ameliorate this limitation by developing SERENGETI, a massively multilingual language model that covers 517 African languages and language varieties. We evaluate our novel models on eight natural language understanding tasks across 20 datasets, comparing to 4 mPLMs that cover 4-23 African languages. SERENGETI outperforms other models on 11 datasets across the eights tasks, achieving 82.27 average F1. We also perform analyses of errors from our models, which allows us to investigate the influence of language genealogy and linguistic similarity when the models are applied under zero-shot settings. We will publicly release our models for research.footnotehrefhttps://github.com/UBC-NLP/serengetihttps://github.com/UBC-NLP/serengeti

Key facts. Cited by 40 papers in the map (data through 14 Oct 2026). Builds on 45 of its 128 references that are in the map. First posted on arXiv 21 Dec 2022.

Builds on

45 of 128 references.

References · 45

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  6. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  7. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  8. 2018Deep contextualized word representationscited by 2,290
  9. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  10. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  11. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
  12. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,035
33 more
  1. 2016Semantics derived automatically from language corpora contain human-like biasescited by 901
  2. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 872
  3. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 798
  4. 2019On the Cross-lingual Transferability of Monolingual Representationscited by 578
  5. 2020Beyond English-Centric Multilingual Machine Translationcited by 561
  6. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 488
  7. 2020XCOPA: A Multilingual Dataset for Causal Commonsense Reasoningcited by 333
  8. 2019What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language modelscited by 320
  9. 2021Quality at a Glance: An Audit of Web-Crawled Multilingual Datasetscited by 243
  10. 2021MasakhaNER: Named Entity Recognition for African Languagescited by 155
  11. 2022Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuningcited by 149
  12. 2018Multilingual Constituency Parsing with Self-Attention and Pre-Trainingcited by 117
  13. 2020Fast WordPiece Tokenizationcited by 87
  14. 2021Larger-Scale Transformers for Multilingual Masked Language Modelingcited by 87
  15. 2021XLM-E: Cross-lingual Language Model Pre-training via ELECTRAcited by 72
  16. 2022One Country, 700+ Languages: NLP Challenges for Underrepresented Languages and Dialects in Indonesiacited by 66
  17. 2022NaijaSenti: A Nigerian Twitter Sentiment Corpus for Multilingual Sentiment Analysiscited by 64
  18. 2020Transfer Learning and Distant Supervision for Multilingual Transformer Models: A Study on African Languagescited by 52
  19. 2021A Primer on Pretrained Multilingual Language Modelscited by 49
  20. 2022Towards Afrocentric NLP for African Languages: Where We Are and Where We Can Gocited by 47
  21. 2022AfroLM: A Self-Active Learning-based Multilingual Pretrained Language Model for 23 African Languagescited by 46
  22. 2022AfroLID: A Neural Language Identification Tool for African Languagescited by 39
  23. 2022T-NER: An All-Round Python Library for Transformer-based Named Entity Recognitioncited by 37
  24. 2019Massive vs. Curated Word Embeddings for Low-Resourced Languages. The Case of Yor`ub'a and Twicited by 33
  25. 2020KINNEWS and KIRNEWS: Benchmarking Cross-Lingual Text Classification for Kinyarwanda and Kirundicited by 31
  26. 2020Diversity and Inclusion Metrics in Subset Selectioncited by 27
  27. 2022KinyaBERT: a Morphology-aware Kinyarwanda Language Modelcited by 27
  28. 2021Improved Language Identification Through Cross-Lingual Self-Supervised Learningcited by 20
  29. 2022yosm: A new yoruba sentiment corpus for movie reviewscited by 14
  30. 2020Semantic Enrichment of Nigerian Pidgin English for Contextual Sentiment Classificationcited by 12
  31. 2021An Amharic News Text classification Datasetcited by 10
  32. 2020Measuring Diversity of Artificial Intelligence Conferencescited by 5
  33. 2021Bambara Language Dataset for Sentiment Analysiscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 9; 2024: 12; 2025: 13; 2026: 6.

40 papers.

Sort:

By area:

  1. 2023MADLAD-400: A Multilingual And Document-Level Large Audited Datasetcited by 181
  2. 2023SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialectscited by 152
  3. 2023Glot500: Scaling Multilingual Corpora and Language Models to 500 Languagescited by 116
  4. 2024CLIcK: A Benchmark Dataset of Cultural and Linguistic Intelligence in Koreancited by 66
  5. 2024IrokoBench: A New Benchmark for African Languages in the Age of Large Language Modelscited by 43
  6. 2023SemEval-2023 Task 12: Sentiment Analysis for African Languages (AfriSenti-SemEval)cited by 41
  7. 2024Goldfish: Monolingual Language Models for 350 Languagescited by 29
  8. 2024MaLA-500: Massive Language Adaptation of Large Language Modelscited by 28
  9. 2023Taxi1500: A Multilingual Dataset for Text Classification in 1500 Languagescited by 23
  10. 2024GlotCC: An Open Broad-Coverage CommonCrawl Corpus and Pipeline for Minority Languagescited by 16
  11. 2025An Expanded Massive Multilingual Dataset for High-Performance Language Technologies (HPLT)cited by 16
  12. 2025Charting the Landscape of African NLP: Mapping Progress and Shaping the Road Aheadcited by 16
28 more
  1. 2025Where Are We? Evaluating LLM Performance on African Languagescited by 10
  2. 2023How Good are Commercial Large Language Models on African Languages?cited by 8
  3. 2024EthioLLM: Multilingual Large Language Models for Ethiopian Languages with Task Evaluationcited by 8
  4. 2024Africa-Centric Self-Supervised Pre-Training for Multilingual Speech Representation in a Sub-Saharan Contextcited by 8
  5. 2025Lugha-Llama: Adapting Large Language Models for African Languagescited by 8
  6. 2024State of NLP in Kenya: A Surveycited by 7
  7. 2023PuoBERTa: Training and evaluation of a curated language model for Setswanacited by 6
  8. 2025Ibom NLP: A Step Toward Inclusive Natural Language Processing for Nigeria's Minority Languagescited by 6
  9. 2024A Systematic Review of NLP for Ghanaian Languages: Datasets, Models, and a Research Roadmapcited by 5
  10. 2023NaijaRC: A Multi-choice Reading Comprehension Dataset for Nigerian Languagescited by 4
  11. 2025Voice of a Continent: Mapping Africa's Speech Technology Frontiercited by 4
  12. 2024TransliCo: A Contrastive Learning Framework to Address the Script Barrier in Multilingual Pretrained Language Modelscited by 3
  13. 2025Optimized Text Embedding Models and Benchmarks for Amharic Passage Retrievalcited by 3
  14. 2026Can Embedding Similarity Predict Cross-Lingual Transfer? A Systematic Study on African Languagescited by 3
  15. 2024ANGOFA: Leveraging OFA Embedding Initialization and Synthetic Data for Angolan Language Modelcited by 2
  16. 2024From N-grams to Pre-trained Multilingual Models For Language Identificationcited by 2
  17. 2025Bridging Gaps in Natural Language Processing for Yor`ub'a: A Systematic Review of a Decade of Progress and Prospectscited by 2
  18. 2025Improving Multilingual Math Reasoning for African Languagescited by 1
  19. 2025Towards culturally-appropriate conversational AI for health in the majority world: An exploratory study with citizens and professionals in Latin Americacited by 1
  20. 2026QQ: A Language Metadata Toolkit for Multilingual NLPcited by 1
  21. 2023UBC-DLNLP at SemEval-2023 Task 12: Impact of Transfer Learning on African Sentiment Analysiscited by 0
  22. 2025The African Languages Lab: A Collaborative Approach to Advancing Low-Resource African NLPcited by 0
  23. 2025Safe in the Future, Dangerous in the Past: Dissecting Temporal and Linguistic Vulnerabilities in LLMscited by 0
  24. 2025Opportunities and Challenges of Natural Language Processing for Low-Resource Senegalese Languages in Social Science Researchcited by 0
  25. 2026AfroScope: A Framework for Studying the Linguistic Landscape of Africacited by 0
  26. 2026Culturally-Grounded Governance for Multilingual Language Models: Rights, Data Boundaries, and Accountable AI Designcited by 0
  27. 2026Budget-Xfer: Budget-Constrained Source Language Selection for Cross-Lingual Transfer to African Languagescited by 0
  28. 2026When Does Data Augmentation Help? Evaluating LLM and Back-Translation Methods for Hausa and Fongbe NLPcited by 0