# High-Resource Methodological Bias in Low-Resource Investigations

arXiv: https://arxiv.org/abs/2211.07534 · HTML: https://mlsanity.com/paper/2211.07534/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 81 references that are in the map.
- First posted on arXiv 14 Nov 2022.

## Builds on (most cited first, 30 of 35)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [How multilingual is Multilingual BERT?](https://mlsanity.com/paper/1906.01502/) (2019)
- [A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios](https://mlsanity.com/paper/2010.12309/) (2020)
- [MasakhaNER: Named Entity Recognition for African Languages](https://mlsanity.com/paper/2103.11811/) (2021)
- [Data Augmentation for Low-Resource Neural Machine Translation](https://mlsanity.com/paper/1705.00440/) (2017)
- [Meta-Learning for Low-Resource Neural Machine Translation](https://mlsanity.com/paper/1808.08437/) (2018)
- [Survey of Low-Resource Machine Translation](https://mlsanity.com/paper/2109.00486/) (2021)
- [What's in the Box? A Preliminary Analysis of Undesirable Content in the Common Crawl Corpus](https://mlsanity.com/paper/2105.02732/) (2021)
- [Language ID in the Wild: Unexpected Challenges on the Path to a Thousand-Language Web Text Corpus](https://mlsanity.com/paper/2010.14571/) (2020)
- [Transfer Learning and Distant Supervision for Multilingual Transformer Models: A Study on African Languages](https://mlsanity.com/paper/2010.03179/) (2020)
- [Contextual Parameter Generation for Universal Neural Machine Translation](https://mlsanity.com/paper/1808.08493/) (2018)
- [Dual Conditional Cross-Entropy Filtering of Noisy Parallel Corpora](https://mlsanity.com/paper/1809.00197/) (2018)
- [Does the Order of Training Samples Matter? Improving Neural Data-to-Text Generation with Curriculum Learning](https://mlsanity.com/paper/2102.03554/) (2021)
- [Multilingual Part-of-Speech Tagging with Bidirectional Long Short-Term Memory Models and Auxiliary Loss](https://mlsanity.com/paper/1604.05529/) (2016)
- [Cross-Lingual Abstractive Summarization with Limited Parallel Resources](https://mlsanity.com/paper/2105.13648/) (2021)
- [Optimizing Transformer for Low-Resource Neural Machine Translation](https://mlsanity.com/paper/2011.02266/) (2020)
- [Key Fact as Pivot: A Two-Stage Model for Low Resource Table-to-Text Generation](https://mlsanity.com/paper/1908.03067/) (2019)
- [Machine Translation into Low-resource Language Varieties](https://mlsanity.com/paper/2106.06797/) (2021)
- [Reusing a Pretrained Language Model on Languages with Limited Corpora for Unsupervised NMT](https://mlsanity.com/paper/2009.07610/) (2020)
- [Parallel Corpus Filtering via Pre-trained Language Models](https://mlsanity.com/paper/2005.06166/) (2020)
- [Endangered Languages are not Low-Resourced!](https://mlsanity.com/paper/2103.09567/) (2021)
- [Conversations with Documents. An Exploration of Document-Centered Assistance](https://mlsanity.com/paper/2002.00747/) (2020)
- [On the Importance of Subword Information for Morphological Tasks in Truly Low-Resource Languages](https://mlsanity.com/paper/1909.12375/) (2019)
- [Reducing Confusion in Active Learning for Part-Of-Speech Tagging](https://mlsanity.com/paper/2011.00767/) (2020)

## Built on by (most cited first, 1 of 1)

- [Opportunities and Challenges of Large Language Models for Low-Resource Languages in Humanities Research](https://mlsanity.com/paper/2412.04497/) (2024)

## Abstract (arXiv)

The central bottleneck for low-resource NLP is typically regarded to be the quantity of accessible data, overlooking the contribution of data quality. This is particularly seen in the development and evaluation of low-resource systems via down sampling of high-resource language data. In this work we investigate the validity of this approach, and we specifically focus on two well-known NLP tasks for our empirical investigations: POS-tagging and machine translation. We show that down sampling from a high-resource language results in datasets with different properties than the low-resource datasets, impacting the model performance for both POS-tagging and machine translation. Based on these results we conclude that naive down sampling of datasets results in a biased view of how well these systems work in a low-resource scenario.