mlsanityData to 7 Oct 2026

Toward Deconfounding the Influence of Entity Demographics for Question Answering Accuracy

arXiv 2104.07571 ↗First posted 15 Apr 2021cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

The goal of question answering (QA) is to answer any question. However, major QA datasets have skewed distributions over gender, profession, and nationality. Despite that skew, model accuracy analysis reveals little evidence that accuracy is lower for people based on gender or nationality; instead, there is more variation on professions (question topic). But QA's lack of representation could itself hide evidence of bias, necessitating QA datasets that better represent global diversity.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 15 of its 38 references that are in the map. First posted on arXiv 15 Apr 2021.

Builds on

15 of 38 references.

References · 15

  1. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  2. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  3. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  4. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 657
  5. 2019On the Cross-lingual Transferability of Monolingual Representationscited by 575
  6. 2017Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraintscited by 420
  7. 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 385
  8. 2019MLQA: Evaluating Cross-lingual Extractive Question Answeringcited by 339
  9. 2019Evaluating Gender Bias in Machine Translationcited by 247
  10. 2020UnQovering Stereotyping Biases via Underspecified Questionscited by 124
  11. 2020Measuring Social Biases in Grounded Vision and Language Embeddingscited by 61
  12. 2018What Makes Reading Comprehension Questions Easier?cited by 59
3 more
  1. 2020What do Models Learn from Question Answering Datasets?cited by 45
  2. 2019A BERT Baseline for the Natural Questionscited by 33
  3. 2019What Question Answering can Learn from Trivia Nerdscited by 21

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 2; 2024: 0; 2025: 1.

5 papers.

  1. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  2. 2023NLPositionality: Characterizing Design Biases of Datasets and Modelscited by 91
  3. 2022Characteristics of Harmful Text: Towards Rigorous Benchmarking of Language Modelscited by 49
  4. 2023Mitigating Bias for Question Answering Models by Tracking Bias Influencecited by 8
  5. 2025Open-DeBias: Toward Mitigating Open-Set Bias in Language Modelscited by 0