mlsanityData to 7 Oct 2026

Look at the First Sentence: Position Bias in Question Answering

arXiv 2004.14602 ↗First posted 30 Apr 2020cs.CLcited by 83

Not on the map. Why?

Abstract (arXiv)

Many extractive question answering models are trained to predict start and end positions of answers. The choice of predicting answers as positions is mainly due to its simplicity and effectiveness. In this study, we hypothesize that when the distribution of the answer positions is highly skewed in the training set (e.g., answers lie only in the k-th sentence of each passage), QA models predicting answers as positions can learn spurious positional cues and fail to give answers in different positions. We first illustrate this position bias in popular extractive QA models such as BiDAF and BERT and thoroughly examine how position bias propagates through each layer of BERT. To safely deliver position information without position bias, we train models with various de-biasing methods including entropy regularization and bias ensembling. Among them, we found that using the prior distribution of answer positions as a bias model is very effective at reducing position bias, recovering the performance of BERT from 37.48% to 81.64% when trained on a biased SQuAD dataset.

Key facts. Cited by 83 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 14; 2023: 18; 2024: 28; 2025: 8; 2026: 7.

83 papers.

Sort:

By area:

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  2. 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 451
  3. 2023JudgeLM: Fine-tuned Large Language Models are Scalable Judgescited by 351
  4. 2023Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicinecited by 319
  5. 2023From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuningcited by 301
  6. 2024Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuningcited by 134
  7. 2021KLUE: Korean Language Understanding Evaluationcited by 113
  8. 2022Shortcut Learning of Large Language Models in Natural Language Understandingcited by 83
  9. 2024Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuningcited by 57
  10. 2021Semantic Answer Similarity for Evaluating Question Answering Modelscited by 47
  11. 2026EvoScientist: Towards Multi-Agent Evolving AI Scientists for End-to-End Scientific Discoverycited by 41
  12. 2022Can Rationalization Improve Robustness?cited by 36
38 more
  1. 2024Can LLMs Speak For Diverse People? Tuning LLMs via Debate to Generate Controllable Controversial Statementscited by 33
  2. 2022The Tail Wagging the Dog: Dataset Construction Biases of Social Bias Benchmarkscited by 31
  3. 2024Small Language Models Learn Enhanced Reasoning Skills from Medical Textbookscited by 30
  4. 2023Reflection-Tuning: Data Recycling Improves LLM Instruction-Tuningcited by 26
  5. 2021Introspective Distillation for Robust Question Answeringcited by 25
  6. 2021Analyzing Dynamic Adversarial Training Data in the Limitcited by 24
  7. 2024From Medprompt to o1: Exploration of Run-Time Strategies for Medical Challenge Problems and Beyondcited by 24
  8. 2022MERIt: Meta-Path Guided Contrastive Learning for Logical Reasoningcited by 22
  9. 2024Debatrix: Multi-dimensional Debate Judge with Iterative Chronological Analysis Based on LLMcited by 21
  10. 2023ExtractGPT: Exploring the Potential of Large Language Models for Product Attribute Value Extractioncited by 20
  11. 2024MACPO: Weak-to-Strong Alignment via Multi-Agent Contrastive Preference Optimizationcited by 20
  12. 2023AlpaGasus: Training A Better Alpaca with Fewer Datacited by 19
  13. 2022Saving Dense Retriever from Shortcut Dependency in Conversational Searchcited by 18
  14. 2024Look, Listen, and Answer: Overcoming Biases for Audio-Visual Question Answeringcited by 18
  15. 2022Reducing Position Bias in Simultaneous Machine Translation with Length-Aware Frameworkcited by 16
  16. 2025Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associationscited by 14
  17. 2022Machine Reading, Fast and Slow: When Do Models "Understand" Language?cited by 13
  18. 2024ULTRA: Unleash LLMs' Potential for Event Argument Extraction through Hierarchical Modeling and Pair-wise Self-Refinementcited by 12
  19. 2021Exploring Strategies for Generalizable Commonsense Reasoning with Pre-trained Modelscited by 11
  20. 2024Analyzing the Evaluation of Cross-Lingual Knowledge Transfer in Multilingual Language Modelscited by 11
  21. 2024KnowTuning: Knowledge-aware Fine-tuning for Large Language Modelscited by 11
  22. 2024RuleR: Improving LLM Controllability by Rule-based Data Recyclingcited by 11
  23. 2023Guiding LLM to Fool Itself: Automatically Manipulating Machine Reading Comprehension Shortcut Triggerscited by 10
  24. 2022Which Shortcut Solution Do Question Answering Models Prefer to Learn?cited by 9
  25. 2024Mosaic-IT: Cost-Free Compositional Data Synthesis for Instruction Tuningcited by 9
  26. 2021Can Question Generation Debias Question Answering Models? A Case Study on Question-Context Lexical Overlapcited by 8
  27. 2022longhorns at DADC 2022: How many linguists does it take to fool a Question Answering model? A systematic approach to adversarial attackscited by 8
  28. 2024Shortcut Learning in In-Context Learning: A Surveycited by 8
  29. 2021How Do Your Biomedical Named Entity Recognition Models Generalize to Novel Entities?cited by 7
  30. 2022Look to the Right: Mitigating Relative Position Bias in Extractive Question Answeringcited by 7
  31. 2023Think Twice: Measuring the Efficiency of Eliminating Prediction Shortcuts of Question Answering Modelscited by 6
  32. 2024Where is the answer? Investigating Positional Bias in Language Model Knowledge Extractioncited by 6
  33. 2024IryoNLP at MEDIQA-CORR 2024: Tackling the Medical Error Detection & Correction Task On the Shoulders of Medical Agentscited by 6
  34. 2024IterSelectTune: An Iterative Training Framework for Efficient Instruction-Tuning Data Selectioncited by 6
  35. 2025Add-One-In: Incremental Sample Selection for Large Language Models via a Choice-Based Greedy Paradigmcited by 6
  36. 2021Regularization for Long Named Entity Recognitioncited by 5
  37. 2022A Survey on Measuring and Mitigating Reasoning Shortcuts in Machine Reading Comprehensioncited by 4
  38. 2022NAPG: Non-Autoregressive Program Generation for Hybrid Tabular-Textual Question Answeringcited by 4

50 most cited of 83 All 83 (JSON)