mlsanityData to 7 Oct 2026

Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets

arXiv 1908.07898 ↗First posted 21 Aug 2019cs.CLcited by 173

Not on the map. Why?

Abstract (arXiv)

Crowdsourcing has been the prevalent paradigm for creating natural language understanding datasets in recent years. A common crowdsourcing practice is to recruit a small number of high-quality workers, and have them massively generate examples. Having only a few workers generate the majority of examples raises concerns about data diversity, especially when workers freely generate sentences. In this paper, we perform a series of experiments showing these concerns are evident in three recent NLP datasets. We show that model performance improves when training with annotator identifiers as features, and that models are able to recognize the most productive annotators. Moreover, we show that often models do not generalize well to examples from annotators that did not contribute to the training set. Our findings suggest that annotator bias should be monitored during dataset creation, and that test set annotators should be disjoint from training set annotators.

Key facts. Cited by 173 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Aug 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 56; 2022: 36; 2023: 35; 2024: 25; 2025: 9; 2026: 12.

173 papers.

Sort:

By area:

  1. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  2. 2021Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotationscited by 306
  3. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 292
  4. 2021Symbolic Knowledge Distillation: from General Language Models to Commonsense Modelscited by 270
  5. 2021Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detectioncited by 258
  6. 2023From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Modelscited by 257
  7. 2022WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creationcited by 200
  8. 2024Are We Done with MMLU?cited by 178
  9. 2021MultiModalQA: Complex Question Answering over Text, Tables and Imagescited by 177
  10. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 129
  11. 2021Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processingcited by 126
  12. 2021Reframing Human-AI Collaboration for Generating Free-Text Explanationscited by 125
38 more
  1. 2021Gender Bias in Machine Translationcited by 116
  2. 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
  3. 2021Measure and Improve Robustness in NLP Models: A Surveycited by 107
  4. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 97
  5. 2021Competency Problems: On Finding and Removing Artifacts in Language Datacited by 91
  6. 2021QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehensioncited by 91
  7. 2022DataPerf: Benchmarks for Data-Centric AI Developmentcited by 89
  8. 2021CREAK: A Dataset for Commonsense Reasoning over Entity Knowledgecited by 88
  9. 2022Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Surveycited by 80
  10. 2021Improving Question Answering Model Robustness with Synthetic Adversarial Data Generationcited by 77
  11. 2024MALT: Improving Reasoning with Multi-Agent LLM Trainingcited by 71
  12. 2024Artifacts or Abduction: How Do LLMs Answer Multiple-Choice Questions Without the Question?cited by 68
  13. 2021Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristicscited by 67
  14. 2021"Will You Find These Shortcuts?" A Protocol for Evaluating the Faithfulness of Input Salience Methods for Text Classificationcited by 66
  15. 2022A Survey on Programmatic Weak Supervisioncited by 66
  16. 2023Uncertainty in Natural Language Generation: From Theory to Applicationscited by 61
  17. 2022Generating Data to Mitigate Spurious Correlations in Natural Language Inference Datasetscited by 54
  18. 2021Dynaboard: An Evaluation-As-A-Service Platform for Holistic Next-Generation Benchmarkingcited by 53
  19. 2023The Past, Present and Better Future of Feedback Learning in Large Language Models for Subjective Human Preferences and Valuescited by 53
  20. 2021Changing the World by Changing the Datacited by 51
  21. 2022Multilingual HateCheck: Functional Tests for Multilingual Hate Speech Detection Modelscited by 51
  22. 2021Semantic Answer Similarity for Evaluating Question Answering Modelscited by 47
  23. 2022Don't Blame the Annotator: Bias Already Starts in the Annotation Instructionscited by 47
  24. 2021Prompting Contrastive Explanations for Commonsense Reasoning Taskscited by 45
  25. 2023Measuring and Improving Chain-of-Thought Reasoning in Vision-Language Modelscited by 45
  26. 2021COM2SENSE: A Commonsense Reasoning Benchmark with Complementary Sentencescited by 40
  27. 2023You Are What You Annotate: Towards Better Models through Annotator Representationscited by 40
  28. 2022Large-Scale Hate Speech Detection with Cross-Domain Transfercited by 39
  29. 2023Analyzing Dataset Annotation Quality Management in the Wildcited by 39
  30. 2024The Perspectivist Paradigm Shift: Assumptions and Challenges of Capturing Human Labelscited by 37
  31. 2021SD-QA: Spoken Dialectal Question Answering for the Real Worldcited by 36
  32. 2021Perceptual Score: What Data Modalities Does Your Model Perceive?cited by 35
  33. 2021Back to Square One: Artifact Detection, Training and Commonsense Disentanglement in the Winograd Schemacited by 32
  34. 2021Models in the Loop: Aiding Crowdworkers with Generative Annotation Assistantscited by 31
  35. 2023Mind the instructions: a holistic evaluation of consistency and interactions in prompt-based learningcited by 31
  36. 2024Measuring and Reducing LLM Hallucination without Gold-Standard Answerscited by 31
  37. 2024Human and LLM Biases in Hate Speech Annotations: A Socio-Demographic Analysis of Annotators and Targetscited by 31
  38. 2021Automatic Generation of Contrast Sets from Scene Graphs: Probing the Compositional Consistency of GQAcited by 30

50 most cited of 173 All 173 (JSON)