MasakhaNER: Named Entity Recognition for African Languages
Not on the map. Why?
Abstract (arXiv)
We take a step towards addressing the under-representation of the African continent in NLP research by creating the first large publicly available high-quality dataset for named entity recognition (NER) in ten African languages, bringing together a variety of stakeholders. We detail characteristics of the languages to help researchers understand the challenges that these languages pose for NER. We analyze our datasets and conduct an extensive empirical evaluation of state-of-the-art methods across both supervised and transfer learning settings. We release the data, code, and models in order to inspire future research on African NLP.
Key facts. Cited by 155 papers in the map (data through 7 Oct 2026). Builds on 27 of its 77 references that are in the map. First posted on arXiv 22 Mar 2021.
Builds on
27 of 77 references.
References · 27
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
- 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
- 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
- 2016Enriching Word Vectors with Subword Informationcited by 1,578
- 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
- 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 738
- 2016Neural Architectures for Named Entity Recognitioncited by 548
- 2015Bidirectional LSTM-CRF Models for Sequence Taggingcited by 544
- 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 443
15 more
- 2016End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRFcited by 341
- 2020LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attentioncited by 326
- 2015Named Entity Recognition with Bidirectional LSTM-CNNscited by 201
- 2020Participatory Research for Low-resourced Machine Translation: A Case Study in African Languagescited by 159
- 2019CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairscited by 136
- 2019A Survey on Recent Advances in Named Entity Recognition from Deep Learning modelscited by 117
- 2020UNKs Everywhere: Adapting Multilingual Language Models to New Scriptscited by 113
- 2020Transfer Learning and Distant Supervision for Multilingual Transformer Models: A Study on African Languagescited by 52
- 2019Massive vs. Curated Word Embeddings for Low-Resourced Languages. The Case of Yor`ub'a and Twicited by 33
- 2020KINNEWS and KIRNEWS: Benchmarking Cross-Lingual Text Classification for Kinyarwanda and Kirundicited by 31
- 2020Interpretable Multi-dataset Evaluation for Named Entity Recognitioncited by 27
- 2017DyNet: The Dynamic Neural Network Toolkitcited by 21
- 2020Soft Gazetteers for Low-Resource Named Entity Recognitioncited by 20
- 2019A Focus on Neural Machine Translation for African Languagescited by 15
- 2020Igbo-English Machine Translation: An Evaluation Benchmarkcited by 14
Built on by
Citing papers per year: 2021: 5; 2022: 33; 2023: 45; 2024: 28; 2025: 29; 2026: 15.
155 papers.
Sort:
By area:
- 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 292
- 2022Challenges and Strategies in Cross-Cultural NLPcited by 216
- 2023The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Datasetcited by 148
- 2021Composable Sparse Fine-Tuning for Cross-Lingual Transfercited by 144
- 2022A Few Thousand Translations Go a Long Way! Leveraging Pre-trained Models for African News Translationcited by 97
- 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 97
- 2023NLPositionality: Characterizing Design Biases of Datasets and Modelscited by 91
- 2023XLM-V: Overcoming the Vocabulary Bottleneck in Multilingual Masked Language Modelscited by 88
- 2023AfriSenti: A Twitter Sentiment Analysis Benchmark for African Languagescited by 68
- 2023BUFFET: Benchmarking Large Language Models for Few-shot Cross-lingual Transfercited by 68
- 2022NusaX: Multilingual Parallel Sentiment Dataset for 10 Indonesian Local Languagescited by 67
- 2022NaijaSenti: A Nigerian Twitter Sentiment Corpus for Multilingual Sentiment Analysiscited by 64
38 more
- 2024What Matters for Model Merging at Scale?cited by 52
- 2023LLM-powered Data Augmentation for Enhanced Cross-lingual Performancecited by 51
- 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontierscited by 49
- 2022AfroLM: A Self-Active Learning-based Multilingual Pretrained Language Model for 23 African Languagescited by 46
- 2022Expanding Pretrained Models to Thousands More Languages via Lexicon-based Adaptationcited by 44
- 2023Improving Language Plasticity via Pretraining with Active Forgettingcited by 43
- 2023AfroBench: How Good are Large Language Models on African Languages?cited by 43
- 2024IrokoBench: A New Benchmark for African Languages in the Age of Large Language Modelscited by 43
- 2023SemEval-2023 Task 12: Sentiment Analysis for African Languages (AfriSenti-SemEval)cited by 41
- 2024SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languagescited by 41
- 2022SERENGETI: Massively Multilingual Language Models for Africacited by 40
- 2022MasakhaNER 2.0: Africa-centric Transfer Learning for Named Entity Recognitioncited by 38
- 2022NusaCrowd: Open Source Initiative for Indonesian NLP Resourcescited by 38
- 2023XTREME-UP: A User-Centric Scarce-Data Benchmark for Under-Represented Languagescited by 34
- 2023When Is Multilinguality a Curse? Language Modeling for 250 High- and Low-Resource Languagescited by 32
- 2023MasakhaNEWS: News Topic Classification for African languagescited by 30
- 2022KinyaBERT: a Morphology-aware Kinyarwanda Language Modelcited by 27
- 2022What a Creole Wants, What a Creole Needscited by 27
- 2024Goldfish: Monolingual Language Models for 350 Languagescited by 27
- 2023Universal NER: A Gold-Standard Multilingual Named Entity Recognition Benchmarkcited by 26
- 2022Beyond Counting Datasets: A Survey of Multilingual Dataset Construction and Necessary Resourcescited by 25
- 2022GreenPLM: Cross-Lingual Transfer of Monolingual Pre-Trained Language Models at Almost No Costcited by 24
- 2023AfriQA: Cross-lingual Open-Retrieval Question Answering for African Languagescited by 23
- 2023MasakhaPOS: Part-of-Speech Tagging for Typologically Diverse African Languagescited by 23
- 2021Dataset Geography: Mapping Language Data to Language Userscited by 22
- 2025Beyond English: The Impact of Prompt Translation Strategies across Languages and Tasks in Multilingual LLMscited by 22
- 2023InstructAlign: High-and-Low Resource Language Alignment via Continual Crosslingual Instruction Tuningcited by 21
- 2023A Material Lens on Coloniality in NLPcited by 19
- 2021On Language Models for Creolescited by 17
- 2023Multi3WOZ: A Multilingual, Multi-Domain, Multi-Parallel Dataset for Training and Evaluating Culturally Adapted Task-Oriented Dialog Systemscited by 16
- 2024EMMA-500: Enhancing Massively Multilingual Adaptation of Large Language Modelscited by 16
- 2022Intriguing Properties of Compression on Multilingual Modelscited by 15
- 2022MULTI3NLU++: A Multilingual, Multi-Intent, Multi-Domain Dataset for Natural Language Understanding in Task-Oriented Dialoguecited by 15
- 2023Translation and Fusion Improves Zero-shot Cross-lingual Information Extractioncited by 15
- 2023Cross-Lingual Transfer with Target Language-Ready Task Adapterscited by 15
- 2022Toward More Meaningful Resources for Lower-resourced Languagescited by 14
- 2022yosm: A new yoruba sentiment corpus for movie reviewscited by 14
- 2023NollySenti: Leveraging Transfer Learning and Machine Translation for Nigerian Movie Sentiment Classificationcited by 14
50 most cited of 155 All 155 (JSON)
- 2026How Many Labels Does a Language Need? Annotation Budgets and Cross-Lingual Pooling for African-Language Text Classification29 Sep 2026
- 2026KinyaMed: Seeds, Not Rows -- What a Corpus Requirement Written in the Wrong Unit Fails to Constrain26 Sep 2026
- 2026Structure, Association, and Decision Value: Representation-Based Difficulty Estimation for Adaptive Inference in African-Language NLI19 Aug 2026
- 2026Expanding the Lexicon of Ge'ez Based African Languages: A Comparative Study of Amharic and Tigrinya16 Jul 2026
- 2026Open but Incompatible: A License Compatibility Analysis of Corpora for Low-Resource African Languages27 Jun 2026
- 2026Toward Responsible and Epistemically Grounded Multilingual LLMs for Computational Social Science and Humanities30 May 2026
- 2026SomaliWeb v1: A Quality-Filtered Somali Web Corpus with a Matched Tokenizer and a Public Language-Identification Benchmark18 May 2026
- 2026DeepRefine: Agentic Knowledge Refinement via Reinforcement Learning11 May 2026
- 2026Anthropogenic Regional Adaptation in Multimodal Vision-Language Model13 Apr 2026
- 2026YoNER: A New Yor`ub'a Multi-domain Named Entity Recognition Dataset7 Apr 2026
- 2026Budget-Xfer: Budget-Constrained Source Language Selection for Cross-Lingual Transfer to African Languages29 Mar 2026
- 2026DariMis: Harm-Aware Modeling for Dari Misinformation Detection on YouTube24 Mar 2026
38 more
- 2026LGSE: Lexically Grounded Subword Embedding Initialization for Low-Resource Language Adaptation23 Mar 2026
- 2026An Ubuntu-Guided Large Language Model Framework for Cognitive Behavioral Mental Health Dialogue11 Jan 2026
- 2026Can Embedding Similarity Predict Cross-Lingual Transfer? A Systematic Study on African Languages6 Jan 2026
- 2025Opportunities and Challenges of Natural Language Processing for Low-Resource Senegalese Languages in Social Science Research24 Dec 2025
- 2025Language Diversity: Evaluating Language Usage and AI Performance on African Languages in Digital Spaces1 Dec 2025
- 2025Named Entity Recognition for the Kurdish Sorani Language: Dataset Creation and Comparative Analysis27 Nov 2025
- 2025AfriStereo: A Culturally Grounded Dataset for Evaluating Stereotypical Bias in Large Language Models27 Nov 2025
- 2025Evaluating Machine Translation Datasets for Low-Web Data Languages: A Gendered Lens5 Nov 2025
- 2025From the Rock Floor to the Cloud: A Systematic Survey of State-of-the-Art NLP in Battery Life Cycle31 Oct 2025
- 2025The African Languages Lab: A Collaborative Approach to Advancing Low-Resource African NLP7 Oct 2025
- 2025Scalable multilingual PII annotation for responsible AI in LLMs3 Oct 2025
- 2025Uncertainty in Semantic Language Modeling with PIXELS23 Sep 2025
- 2025WolBanking77: Wolof Banking Speech Intent Classification Dataset23 Sep 2025
- 2025Benchmarking Sociolinguistic Diversity in Swahili NLP: A Taxonomy-Guided Approach6 Aug 2025
- 2025Breaking Physical and Linguistic Borders: Multilingual Federated Prompt Tuning for Low-Resource Languages2 Jul 2025
- 2025Prompt, Translate, Fine-Tune, Re-Initialize, or Instruction-Tune? Adapting LLMs for In-Context Learning in Low-Resource Languages23 Jun 2025
- 2025One Tokenizer To Rule Them All: Emergent Language Plasticity via Multilingual Tokenizers12 Jun 2025
- 2025Learning to Focus: Causal Attention Distillation via Gradient-Guided Token Pruning9 Jun 2025
- 2025Design of intelligent proofreading system for English translation based on CNN and BERT5 Jun 2025
- 2025Automatic Correction of Writing Anomalies in Hausa Texts4 Jun 2025
- 2025The State of Large Language Models for African Languages: Progress and Challenges2 Jun 2025
- 2025Efficient and Asymptotically Unbiased Constrained Decoding for Large Language Models12 Apr 2025
- 2025Lugha-Llama: Adapting Large Language Models for African Languages9 Apr 2025
- 2025LAG-MMLU: Benchmarking Frontier LLM Understanding in Latvian and Giriama14 Mar 2025
- 2025Crowdsource, Crawl, or Generate? Creating SEA-VL, a Multicultural Vision-Language Dataset for Southeast Asia10 Mar 2025
- 2025NaijaNLP: A Survey of Nigerian Low-Resource Languages27 Feb 2025
- 2025Where Are We? Evaluating LLM Performance on African Languages26 Feb 2025
- 2025Bridging Gaps in Natural Language Processing for Yor`ub'a: A Systematic Review of a Decade of Progress and Prospects24 Feb 2025
- 2025Beyond Release: Access Considerations for Generative AI Systems23 Feb 2025
- 2025Beyond English: The Impact of Prompt Translation Strategies across Languages and Tasks in Multilingual LLMs13 Feb 2025
- 2025RideKE: Leveraging Low-Resource, User-Generated Twitter Content for Sentiment and Emotion Detection in Kenyan Code-Switched Dataset10 Feb 2025
- 2025AfriHate: A Multilingual Collection of Hate Speech and Abusive Language Datasets for African Languages14 Jan 2025
- 2024A Multi-way Parallel Named Entity Annotated Corpus for English, Tamil and Sinhala3 Dec 2024
- 2024Tomato, Tomahto, Tomate: Do Multilingual Language Models Understand Based on Subword-Level Semantic Concepts?7 Nov 2024
- 2024State of NLP in Kenya: A Survey13 Oct 2024
- 2024What Matters for Model Merging at Scale?4 Oct 2024
- 2024EMMA-500: Enhancing Massively Multilingual Adaptation of Large Language Models26 Sep 2024
- 2024A Perspective on Literary Metaphor in the Context of Generative AI2 Sep 2024
50 newest of 155 All 155 (JSON)
- 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 292
- 2022Challenges and Strategies in Cross-Cultural NLPcited by 216
- 2023The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Datasetcited by 148
- 2021Composable Sparse Fine-Tuning for Cross-Lingual Transfercited by 144
- 2022A Few Thousand Translations Go a Long Way! Leveraging Pre-trained Models for African News Translationcited by 97
- 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 97
- 2023NLPositionality: Characterizing Design Biases of Datasets and Modelscited by 91
- 2023XLM-V: Overcoming the Vocabulary Bottleneck in Multilingual Masked Language Modelscited by 88
- 2023AfriSenti: A Twitter Sentiment Analysis Benchmark for African Languagescited by 68
- 2023BUFFET: Benchmarking Large Language Models for Few-shot Cross-lingual Transfercited by 68
- 2022NusaX: Multilingual Parallel Sentiment Dataset for 10 Indonesian Local Languagescited by 67
- 2022NaijaSenti: A Nigerian Twitter Sentiment Corpus for Multilingual Sentiment Analysiscited by 64
12 most cited of 148 All 148 (JSON)
- 2026How Many Labels Does a Language Need? Annotation Budgets and Cross-Lingual Pooling for African-Language Text Classification29 Sep 2026
- 2026KinyaMed: Seeds, Not Rows -- What a Corpus Requirement Written in the Wrong Unit Fails to Constrain26 Sep 2026
- 2026Structure, Association, and Decision Value: Representation-Based Difficulty Estimation for Adaptive Inference in African-Language NLI19 Aug 2026
- 2026Expanding the Lexicon of Ge'ez Based African Languages: A Comparative Study of Amharic and Tigrinya16 Jul 2026
- 2026Open but Incompatible: A License Compatibility Analysis of Corpora for Low-Resource African Languages27 Jun 2026
- 2026Toward Responsible and Epistemically Grounded Multilingual LLMs for Computational Social Science and Humanities30 May 2026
- 2026SomaliWeb v1: A Quality-Filtered Somali Web Corpus with a Matched Tokenizer and a Public Language-Identification Benchmark18 May 2026
- 2026YoNER: A New Yor`ub'a Multi-domain Named Entity Recognition Dataset7 Apr 2026
- 2026Budget-Xfer: Budget-Constrained Source Language Selection for Cross-Lingual Transfer to African Languages29 Mar 2026
- 2026DariMis: Harm-Aware Modeling for Dari Misinformation Detection on YouTube24 Mar 2026
- 2026LGSE: Lexically Grounded Subword Embedding Initialization for Low-Resource Language Adaptation23 Mar 2026
- 2026An Ubuntu-Guided Large Language Model Framework for Cognitive Behavioral Mental Health Dialogue11 Jan 2026
12 newest of 148 All 148 (JSON)