mlsanityData to 7 Oct 2026

Transfer Learning and Distant Supervision for Multilingual Transformer Models: A Study on African Languages

arXiv 2010.03179 ↗First posted 7 Oct 2020cs.CL cs.LGcited by 52

Not on the map. Why?

Abstract (arXiv)

Multilingual transformer models like mBERT and XLM-RoBERTa have obtained great improvements for many NLP tasks on a variety of languages. However, recent works also showed that results from high-resource languages could not be easily transferred to realistic, low-resource scenarios. In this work, we study trends in performance for different amounts of available resources for the three African languages Hausa, isiXhosa and Yor`ub'a on both NER and topic classification. We show that in combination with transfer learning or distant supervision, these models can achieve with as little as 10 or 100 labeled sentences the same performance as baselines with much more supervised training data. However, we also find settings where this does not hold. Our discussions and additional experiments on assumptions such as time and hardware restrictions highlight challenges and opportunities in low-resource learning.

Key facts. Cited by 52 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 15; 2023: 13; 2024: 4; 2025: 7; 2026: 3.

52 papers.

Sort:

By area:

  1. 2021XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluationcited by 167
  2. 2021MasakhaNER: Named Entity Recognition for African Languagescited by 155
  3. 2022Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuningcited by 149
  4. 2023BUFFET: Benchmarking Large Language Models for Few-shot Cross-lingual Transfercited by 68
  5. 2022AfroLM: A Self-Active Learning-based Multilingual Pretrained Language Model for 23 African Languagescited by 46
  6. 2022SERENGETI: Massively Multilingual Language Models for Africacited by 40
  7. 2022MasakhaNER 2.0: Africa-centric Transfer Learning for Named Entity Recognitioncited by 38
  8. 2023XTREME-UP: A User-Centric Scarce-Data Benchmark for Under-Represented Languagescited by 34
  9. 2023MasakhaNEWS: News Topic Classification for African languagescited by 30
  10. 2023Noise-Robust Fine-Tuning of Pretrained Language Models via External Guidancecited by 24
  11. 2023AfriQA: Cross-lingual Open-Retrieval Question Answering for African Languagescited by 23
  12. 2022Is BERT Robust to Label Noise? A Study on Learning with Noisy Labels in Text Classificationcited by 21
38 more
  1. 2024InkubaLM: A small language model for low-resource African languagescited by 18
  2. 2021Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systemscited by 16
  3. 2025Charting the Landscape of African NLP: Mapping Progress and Shaping the Road Aheadcited by 15
  4. 2021Analysing the Noise Model Error for Realistic Noisy Label Datacited by 14
  5. 2021Label-Descriptive Patterns and Their Application to Characterizing Classification Errorscited by 10
  6. 2023Traffic-Domain Video Question Answering with Automatic Captioningcited by 10
  7. 2025Where Are We? Evaluating LLM Performance on African Languagescited by 9
  8. 2023How Good are Commercial Large Language Models on African Languages?cited by 8
  9. 2022Meta Self-Refinement for Robust Learning with Weak Supervisioncited by 7
  10. 2022Data-Efficient Strategies for Expanding Hate Speech Detection into Under-Resourced Languagescited by 7
  11. 2023To Translate or Not to Translate: A Systematic Investigation of Translation-Based Cross-Lingual Transfer to Low-Resource Languagescited by 7
  12. 2021The first large scale collection of diverse Hausa language datasetscited by 6
  13. 2022Por Qu'e N~ao Utiliser Alla Spraak? Mixed Training with Gradient Optimization in Few-Shot Cross-Lingual Transfercited by 6
  14. 2021Training Cross-Lingual embeddings for Setswana and Sepedicited by 5
  15. 2021Unraveling Social Perceptions & Behaviors towards Migrants on Twittercited by 4
  16. 2022Learning Action Conditions from Instructional Manuals for Instruction Understandingcited by 4
  17. 2022PInKS: Preconditioned Commonsense Inference with Minimal Supervisioncited by 4
  18. 2023Prompt Learning to Mitigate Catastrophic Forgetting in Cross-lingual Transfer for Open-domain Dialogue Generationcited by 4
  19. 2023Handling Realistic Label Noise in BERT Text Classificationcited by 4
  20. 2023Learning with Noisy Labels by Adaptive Gradient-Based Outlier Removalcited by 4
  21. 2023NaijaRC: A Multi-choice Reading Comprehension Dataset for Nigerian Languagescited by 4
  22. 2024Comparing LLM prompting with Cross-lingual transfer performance on Indigenous and Low-resource Brazilian Languagescited by 4
  23. 2021Deep Learning Transformer Architecture for Named Entity Recognition on Low Resourced Languages: State of the art resultscited by 3
  24. 2026Can Embedding Similarity Predict Cross-Lingual Transfer? A Systematic Study on African Languagescited by 3
  25. 2022Cross-lingual Transfer Learning for Check-worthy Claim Identification over Twittercited by 2
  26. 2023Low-Resource Clickbait Spoiling for Indonesian via Question Answeringcited by 2
  27. 2024AlleNoise: large-scale text classification benchmark dataset with real-world label noisecited by 2
  28. 2025Bridging Gaps in Natural Language Processing for Yor`ub'a: A Systematic Review of a Decade of Progress and Prospectscited by 2
  29. 2025HausaNLP: Current Status, Challenges and Future Directions for Hausa Natural Language Processingcited by 2
  30. 2025LiveCLKTBench: Towards Reliable Evaluation of Cross-Lingual Knowledge Transfer in Multilingual LLMscited by 2
  31. 2022ULF: Unsupervised Labeling Function Correction using Cross-Validation for Weak Supervisioncited by 1
  32. 2022Task-Adaptive Pre-Training for Boosting Learning With Noisy Labels: A Study on Text Classification for African Languagescited by 1
  33. 2022High-Resource Methodological Bias in Low-Resource Investigationscited by 1
  34. 2024Mislabeled examples detection viewed as probing machine learning models: concepts, survey and extensive benchmarkcited by 1
  35. 2025FewTopNER: Integrating Few-Shot Learning with Topic Modeling and Named Entity Recognition in a Multilingual Frameworkcited by 1
  36. 2021Exploiting All Samples in Low-Resource Sentence Classification: Early Stopping and Initialization Parameterscited by 0
  37. 2022Analysing the Greek Parliament Records with Emotion Classificationcited by 0
  38. 2025Weed Out, Then Harvest: Dual Low-Rank Adaptation is an Effective Noisy Label Detector for Noise-Robust Learningcited by 0

50 most cited of 52 All 52 (JSON)