# Low-Resource Cross-Lingual Adaptive Training for Nigerian Pidgin

arXiv: https://arxiv.org/abs/2307.00382 · HTML: https://mlsanity.com/paper/2307.00382/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 27 references that are in the map.
- First posted on arXiv 1 Jul 2023.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuning](https://mlsanity.com/paper/2204.06487/) (2022)
- [NaijaSenti: A Nigerian Twitter Sentiment Corpus for Multilingual Sentiment Analysis](https://mlsanity.com/paper/2201.08277/) (2022)
- [On Language Models for Creoles](https://mlsanity.com/paper/2109.06074/) (2021)
- [SMaLL-100: Introducing Shallow Multilingual Machine Translation Model for Low-Resource Languages](https://mlsanity.com/paper/2210.11621/) (2022)
- [Semantic Enrichment of Nigerian Pidgin English for Contextual Sentiment Classification](https://mlsanity.com/paper/2003.12450/) (2020)
- [PidginUNMT: Unsupervised Neural Machine Translation from West African Pidgin to English](https://mlsanity.com/paper/1912.03444/) (2019)
- [Unsupervised Pidgin Text Generation By Pivoting English Data and Self-Training](https://mlsanity.com/paper/2003.08272/) (2020)
- [Towards Supervised and Unsupervised Neural Machine Translation Baselines for Nigerian Pidgin](https://mlsanity.com/paper/2003.12660/) (2020)
- [Towards End-to-End Training of Automatic Speech Recognition for Nigerian Pidgin](https://mlsanity.com/paper/2010.11123/) (2020)
- [A Survey of Orthographic Information in Machine Translation](https://mlsanity.com/paper/2008.01391/) (2020)
- [Robust Neural Machine Translation: Modeling Orthographic and Interpunctual Variation](https://mlsanity.com/paper/2009.05460/) (2020)

## Built on by (most cited first, 5 of 5)

- [Implicit Discourse Relation Classification For Nigerian Pidgin](https://mlsanity.com/paper/2406.18776/) (2024)
- [Modeling Orthographic Variation Improves NLP Performance for Nigerian Pidgin](https://mlsanity.com/paper/2404.18264/) (2024)
- [Surfacing Subtle Stereotypes: A Multilingual, Debate-Oriented Evaluation of Modern LLMs](https://mlsanity.com/paper/2511.01187/) (2025)
- [Desert Camels and Oil Sheikhs: Arab-Centric Red Teaming of Frontier LLMs](https://mlsanity.com/paper/2410.24049/) (2024)
- [Sometin Beta Pass Notin: Improving Multilingual ASR for Nigerian Languages via Knowledge Distillation](https://mlsanity.com/paper/2605.17710/) (2026)

## Abstract (arXiv)

Developing effective spoken language processing systems for low-resource languages poses several challenges due to the lack of parallel data and limited resources for fine-tuning models. In this work, we target on improving upon both text classification and translation of Nigerian Pidgin (Naija) by collecting a large-scale parallel English-Pidgin corpus and further propose a framework of cross-lingual adaptive training that includes both continual and task adaptive training so as to adapt a base pre-trained model to low-resource languages. Our studies show that English pre-trained language models serve as a stronger prior than multilingual language models on English-Pidgin tasks with up to 2.38 BLEU improvements; and demonstrate that augmenting orthographic data and using task adaptive training with back-translation can have a significant impact on model performance.