mlsanityData to 14 Oct 2026

High-Resource Methodological Bias in Low-Resource Investigations

arXiv 2211.07534 ↗First posted 14 Nov 2022cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

The central bottleneck for low-resource NLP is typically regarded to be the quantity of accessible data, overlooking the contribution of data quality. This is particularly seen in the development and evaluation of low-resource systems via down sampling of high-resource language data. In this work we investigate the validity of this approach, and we specifically focus on two well-known NLP tasks for our empirical investigations: POS-tagging and machine translation. We show that down sampling from a high-resource language results in datasets with different properties than the low-resource datasets, impacting the model performance for both POS-tagging and machine translation. Based on these results we conclude that naive down sampling of datasets results in a biased view of how well these systems work in a low-resource scenario.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 35 of its 81 references that are in the map. First posted on arXiv 14 Nov 2022.

Builds on

35 of 81 references.

References · 35

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  6. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  7. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  8. 2019How multilingual is Multilingual BERT?cited by 645
  9. 2020A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarioscited by 171
  10. 2021MasakhaNER: Named Entity Recognition for African Languagescited by 155
  11. 2017Data Augmentation for Low-Resource Neural Machine Translationcited by 139
  12. 2018Meta-Learning for Low-Resource Neural Machine Translationcited by 105
23 more
  1. 2021Survey of Low-Resource Machine Translationcited by 92
  2. 2021What's in the Box? A Preliminary Analysis of Undesirable Content in the Common Crawl Corpuscited by 80
  3. 2020Language ID in the Wild: Unexpected Challenges on the Path to a Thousand-Language Web Text Corpuscited by 77
  4. 2020Transfer Learning and Distant Supervision for Multilingual Transformer Models: A Study on African Languagescited by 52
  5. 2018Contextual Parameter Generation for Universal Neural Machine Translationcited by 40
  6. 2018Dual Conditional Cross-Entropy Filtering of Noisy Parallel Corporacited by 40
  7. 2021Does the Order of Training Samples Matter? Improving Neural Data-to-Text Generation with Curriculum Learningcited by 34
  8. 2016Multilingual Part-of-Speech Tagging with Bidirectional Long Short-Term Memory Models and Auxiliary Losscited by 30
  9. 2021Cross-Lingual Abstractive Summarization with Limited Parallel Resourcescited by 27
  10. 2020Optimizing Transformer for Low-Resource Neural Machine Translationcited by 24
  11. 2019Key Fact as Pivot: A Two-Stage Model for Low Resource Table-to-Text Generationcited by 20
  12. 2021Machine Translation into Low-resource Language Varietiescited by 19
  13. 2020Reusing a Pretrained Language Model on Languages with Limited Corpora for Unsupervised NMTcited by 18
  14. 2020Parallel Corpus Filtering via Pre-trained Language Modelscited by 15
  15. 2021Endangered Languages are not Low-Resourced!cited by 14
  16. 2020Conversations with Documents. An Exploration of Document-Centered Assistancecited by 12
  17. 2019On the Importance of Subword Information for Morphological Tasks in Truly Low-Resource Languagescited by 11
  18. 2020Reducing Confusion in Active Learning for Part-Of-Speech Taggingcited by 11
  19. 2020Unsupervised Neural Machine Translation for Low-Resource Domains via Meta-Learningcited by 9
  20. 2018Distant Supervision from Disparate Sources for Low-Resource Part-of-Speech Taggingcited by 8
  21. 2020Weakly Supervised POS Taggers Perform Poorly on Truly Low-Resource Languagescited by 7
  22. 2021Analyzing Curriculum Learning for Sentiment Analysis along Task Difficulty, Pacing and Visualization Axescited by 2
  23. 2022Summarization with Graphical Elementscited by 2

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Opportunities and Challenges of Large Language Models for Low-Resource Languages in Humanities Researchcited by 27