Analysing Cross-Lingual Transfer in Low-Resourced African Named Entity Recognition
Not on the map. Why?
Abstract (arXiv)
Transfer learning has led to large gains in performance for nearly all NLP tasks while making downstream models easier and faster to train. This has also been extended to low-resourced languages, with some success. We investigate the properties of cross-lingual transfer learning between ten low-resourced languages, from the perspective of a named entity recognition task. We specifically investigate how much adaptive fine-tuning and the choice of transfer language affect zero-shot transfer performance. We find that models that perform well on a single language often do so at the expense of generalising to others, while models with the best generalisation to other languages suffer in individual language performance. Furthermore, the amount of data overlap between the source and target datasets is a better predictor of transfer performance than either the geographical or genetic distance between the languages.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 26 of its 55 references that are in the map. First posted on arXiv 11 Sep 2023.
Builds on
26 of 55 references.
References · 26
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
- 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
- 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,565
- 2019How multilingual is Multilingual BERT?cited by 645
- 2016Neural Architectures for Named Entity Recognitioncited by 550
- 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 445
- 2019SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimizationcited by 254
- 2019Cross-Lingual Ability of Multilingual BERT: An Empirical Studycited by 182
- 2021MasakhaNER: Named Entity Recognition for African Languagescited by 155
14 more
- 2022Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuningcited by 149
- 2019Choosing Transfer Languages for Cross-Lingual Learningcited by 144
- 2019BERTje: A Dutch BERT Modelcited by 109
- 2022A Few Thousand Translations Go a Long Way! Leveraging Pre-trained Models for African News Translationcited by 98
- 2023Spanish Pre-trained BERT Model and Evaluation Datacited by 81
- 2021AmericasNLI: Evaluating Zero-shot Natural Language Understanding of Pretrained Multilingual Models in Truly Low-resource Languagescited by 50
- 2021A Closer Look at How Fine-tuning Changes BERTcited by 42
- 2022A Balanced Data Approach for Evaluating Cross-Lingual Transfer: Mapping the Linguistic Blood Bankcited by 29
- 2022L3Cube-MahaNER: A Marathi Named Entity Recognition Dataset and BERT modelscited by 21
- 2019Zero-shot Reading Comprehension by Cross-lingual Transfer Learning with Multi-lingual Language Representation Modelcited by 18
- 2019Zero-shot Dependency Parsing with Pre-trained Multilingual Sentence Representationscited by 17
- 2021Analysing The Impact Of Linguistic Features On Cross-Lingual Transfercited by 16
- 2021SeqScore: Addressing Barriers to Reproducible Named Entity Recognition Evaluationcited by 8
- 2021CL-NERIL: A Cross-Lingual Model for NER in Indian Languagescited by 1
Built on by
Citing papers per year: 2024: 1; 2025: 1.
2 papers.