# A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios

arXiv: https://arxiv.org/abs/2010.12309 · HTML: https://mlsanity.com/paper/2010.12309/

## Key facts

- Cited by 171 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Oct 2020.

## Built on by (most cited first, 30 of 171)

- [REFINER: Reasoning Feedback on Intermediate Representations](https://mlsanity.com/paper/2304.01904/) (2023)
- [mGPT: Few-Shot Learners Go Multilingual](https://mlsanity.com/paper/2204.07580/) (2022)
- [A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets](https://mlsanity.com/paper/2305.18486/) (2023)
- [mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Dataset](https://mlsanity.com/paper/2108.13897/) (2021)
- [Neural Machine Translation for Low-Resource Languages: A Survey](https://mlsanity.com/paper/2106.15115/) (2021)
- [An Empirical Survey of Data Augmentation for Limited Data Learning in NLP](https://mlsanity.com/paper/2106.07499/) (2021)
- [A Survey on Multilingual Large Language Models: Corpora, Alignment, and Bias](https://mlsanity.com/paper/2404.00929/) (2024)
- [Some Languages are More Equal than Others: Probing Deeper into the Linguistic Disparity in the NLP World](https://mlsanity.com/paper/2210.08523/) (2022)
- [DIALECTBENCH: A NLP Benchmark for Dialects, Varieties, and Closely-Related Languages](https://mlsanity.com/paper/2403.11009/) (2024)
- [Self-Training with Weak Supervision](https://mlsanity.com/paper/2104.05514/) (2021)
- [Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey](https://mlsanity.com/paper/2104.06951/) (2021)
- [Detect Rumors in Microblog Posts for Low-Resource Domains via Adversarial Contrastive Learning](https://mlsanity.com/paper/2204.08143/) (2022)
- [Anticipatory Music Transformer](https://mlsanity.com/paper/2306.08620/) (2023)
- [Multilingual and Cross-Lingual Intent Detection from Spoken Data](https://mlsanity.com/paper/2104.08524/) (2021)
- [Fill-Up: Balancing Long-Tailed Data with Generative Models](https://mlsanity.com/paper/2306.07200/) (2023)
- [Data Augmentation using Large Language Models: Data Perspectives, Learning Paradigms and Challenges](https://mlsanity.com/paper/2403.02990/) (2024)
- [Opportunities and Challenges of Large Language Models for Low-Resource Languages in Humanities Research](https://mlsanity.com/paper/2412.04497/) (2024)
- [Benchmarking Multimodal AutoML for Tabular Data with Text Fields](https://mlsanity.com/paper/2111.02705/) (2021)
- [RuCoLA: Russian Corpus of Linguistic Acceptability](https://mlsanity.com/paper/2210.12814/) (2022)
- [Exploring Predictive Uncertainty and Calibration in NLP: A Study on the Impact of Method & Data Scarcity](https://mlsanity.com/paper/2210.15452/) (2022)
- [Adaptive conformal classification with noisy labels](https://mlsanity.com/paper/2309.05092/) (2023)
- [Is BERT Robust to Label Noise? A Study on Learning with Noisy Labels in Text Classification](https://mlsanity.com/paper/2204.09371/) (2022)
- [A Survey of Methods for Addressing Class Imbalance in Deep-Learning Based Natural Language Processing](https://mlsanity.com/paper/2210.04675/) (2022)
- [InkubaLM: A small language model for low-resource African languages](https://mlsanity.com/paper/2408.17024/) (2024)
- [Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2104.08570/) (2021)
- [Fairness in Language Models Beyond English: Gaps and Challenges](https://mlsanity.com/paper/2302.12578/) (2023)
- [Charting the Landscape of African NLP: Mapping Progress and Shaping the Road Ahead](https://mlsanity.com/paper/2505.21315/) (2025)
- [Analysing the Noise Model Error for Realistic Noisy Label Data](https://mlsanity.com/paper/2101.09763/) (2021)
- [Voice Conversion Can Improve ASR in Very Low-Resource Settings](https://mlsanity.com/paper/2111.02674/) (2021)
- [Representation Learning for Resource-Constrained Keyphrase Generation](https://mlsanity.com/paper/2203.08118/) (2022)

## Abstract (arXiv)

Deep neural networks and huge language models are becoming omnipresent in natural language applications. As they are known for requiring large amounts of training data, there is a growing body of work to improve the performance in low-resource settings. Motivated by the recent fundamental changes towards neural models and the popular pre-train and fine-tune paradigm, we survey promising approaches for low-resource natural language processing. After a discussion about the different dimensions of data availability, we give a structured overview of methods that enable learning when training data is sparse. This includes mechanisms to create additional labeled data like data augmentation and distant supervision as well as transfer learning settings that reduce the need for target supervision. A goal of our survey is to explain how these methods differ in their requirements as understanding them is essential for choosing a technique suited for a specific low-resource setting. Further key aspects of this work are to highlight open issues and to outline promising directions for future research.