# Analysing Cross-Lingual Transfer in Low-Resourced African Named Entity Recognition

arXiv: https://arxiv.org/abs/2309.05311 · HTML: https://mlsanity.com/paper/2309.05311/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 55 references that are in the map.
- First posted on arXiv 11 Sep 2023.

## Builds on (most cited first, 26 of 26)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [How multilingual is Multilingual BERT?](https://mlsanity.com/paper/1906.01502/) (2019)
- [Neural Architectures for Named Entity Recognition](https://mlsanity.com/paper/1603.01360/) (2016)
- [MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer](https://mlsanity.com/paper/2005.00052/) (2020)
- [SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization](https://mlsanity.com/paper/1911.03437/) (2019)
- [Cross-Lingual Ability of Multilingual BERT: An Empirical Study](https://mlsanity.com/paper/1912.07840/) (2019)
- [MasakhaNER: Named Entity Recognition for African Languages](https://mlsanity.com/paper/2103.11811/) (2021)
- [Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuning](https://mlsanity.com/paper/2204.06487/) (2022)
- [Choosing Transfer Languages for Cross-Lingual Learning](https://mlsanity.com/paper/1905.12688/) (2019)
- [BERTje: A Dutch BERT Model](https://mlsanity.com/paper/1912.09582/) (2019)
- [A Few Thousand Translations Go a Long Way! Leveraging Pre-trained Models for African News Translation](https://mlsanity.com/paper/2205.02022/) (2022)
- [Spanish Pre-trained BERT Model and Evaluation Data](https://mlsanity.com/paper/2308.02976/) (2023)
- [AmericasNLI: Evaluating Zero-shot Natural Language Understanding of Pretrained Multilingual Models in Truly Low-resource Languages](https://mlsanity.com/paper/2104.08726/) (2021)
- [A Closer Look at How Fine-tuning Changes BERT](https://mlsanity.com/paper/2106.14282/) (2021)
- [A Balanced Data Approach for Evaluating Cross-Lingual Transfer: Mapping the Linguistic Blood Bank](https://mlsanity.com/paper/2205.04086/) (2022)
- [L3Cube-MahaNER: A Marathi Named Entity Recognition Dataset and BERT models](https://mlsanity.com/paper/2204.06029/) (2022)
- [Zero-shot Reading Comprehension by Cross-lingual Transfer Learning with Multi-lingual Language Representation Model](https://mlsanity.com/paper/1909.09587/) (2019)
- [Zero-shot Dependency Parsing with Pre-trained Multilingual Sentence Representations](https://mlsanity.com/paper/1910.05479/) (2019)
- [Analysing The Impact Of Linguistic Features On Cross-Lingual Transfer](https://mlsanity.com/paper/2105.05975/) (2021)
- [SeqScore: Addressing Barriers to Reproducible Named Entity Recognition Evaluation](https://mlsanity.com/paper/2107.14154/) (2021)
- [CL-NERIL: A Cross-Lingual Model for NER in Indian Languages](https://mlsanity.com/paper/2111.11815/) (2021)

## Built on by (most cited first, 2 of 2)

- [Bridging Gaps in Natural Language Processing for Yor`ub'a: A Systematic Review of a Decade of Progress and Prospects](https://mlsanity.com/paper/2502.17364/) (2025)
- [Bridging the Gap: Enhancing LLM Performance for Low-Resource African Languages with New Benchmarks, Fine-Tuning, and Cultural Adjustments](https://mlsanity.com/paper/2412.12417/) (2024)

## Abstract (arXiv)

Transfer learning has led to large gains in performance for nearly all NLP tasks while making downstream models easier and faster to train. This has also been extended to low-resourced languages, with some success. We investigate the properties of cross-lingual transfer learning between ten low-resourced languages, from the perspective of a named entity recognition task. We specifically investigate how much adaptive fine-tuning and the choice of transfer language affect zero-shot transfer performance. We find that models that perform well on a single language often do so at the expense of generalising to others, while models with the best generalisation to other languages suffer in individual language performance. Furthermore, the amount of data overlap between the source and target datasets is a better predictor of transfer performance than either the geographical or genetic distance between the languages.