mlsanityData to 7 Oct 2026

Simple Entity-Centric Questions Challenge Dense Retrievers

arXiv 2109.08535 ↗First posted 17 Sep 2021cs.CL cs.IRcited by 140

Not on the map. Why?

Cited by 25 in Retrieval-augmented generation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Open-domain question answering has exploded in popularity recently due to the success of dense retrieval models, which have surpassed sparse models using only a few supervised training examples. However, in this paper, we demonstrate current dense models are not yet the holy grail of retrieval. We first construct EntityQuestions, a set of simple, entity-rich questions based on facts from Wikidata (e.g., "Where was Arve Furset born?"), and observe that dense retrievers drastically underperform sparse methods. We investigate this issue and uncover that dense retrievers can only generalize to common entities unless the question pattern is explicitly observed during training. We discuss two simple solutions towards addressing this critical problem. First, we demonstrate that data augmentation is unable to fix the generalization problem. Second, we argue a more robust passage encoder helps facilitate better question adaptation using specialized question encoders. We hope our work can shed light on the challenges in creating a robust, universal dense retriever that works well across different input distributions.

Key facts. Cited by 140 papers in the map (data through 7 Oct 2026). Builds on 23 of its 34 references that are in the map. First posted on arXiv 17 Sep 2021.

Builds on

23 of 34 references.

References · 23

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  3. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  4. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  5. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  6. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  7. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
  8. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  9. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  10. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 410
  11. 2020Autoregressive Entity Retrievalcited by 333
  12. 2020LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attentioncited by 326
11 more
  1. 2019Measuring Bias in Contextualized Word Representationscited by 275
  2. 2021PAQ: 65 Million Probably-Asked Questions and What You Can Do With Themcited by 198
  3. 2019End-to-End Open-Domain Question Answering with BERTserinicited by 152
  4. 2020Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasetscited by 137
  5. 2019Assessing Social and Intersectional Biases in Contextualized Word Representationscited by 134
  6. 2020Entities as Experts: Sparse Memory Access with Entity Supervisioncited by 100
  7. 2020Learning Dense Representations of Phrases at Scalecited by 91
  8. 2019Scalable Zero-shot Entity Linking with Dense Entity Retrievalcited by 85
  9. 2020Efficient One-Pass End-to-End Entity Linking for Questionscited by 76
  10. 2021Pyserini: An Easy-to-Use Python Toolkit to Support Replicable IR Research with Sparse and Dense Representationscited by 60
  11. 2021Evaluating Entity Disambiguation and the Role of Popularity in Retrieval-Based NLPcited by 30

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 25; 2023: 22; 2024: 35; 2025: 26; 2026: 26.

140 papers.

Sort:

By area:

  1. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  2. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  3. 2024Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?cited by 227
  4. 2023Head-to-Tail: How Knowledgeable are Large Language Models (LLMs)? A.K.A. Will LLMs Replace Knowledge Graphs?cited by 167
  5. 2022Improving Passage Retrieval with Zero-Shot Question Generationcited by 164
  6. 2023Poisoning Retrieval Corpora by Injecting Adversarial Passagescited by 153
  7. 2022StreamingQA: A Benchmark for Adaptation to New Knowledge over Time in Question Answering Modelscited by 123
  8. 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 116
  9. 2023Dense X Retrieval: What Retrieval Granularity Should We Use?cited by 88
  10. 2023Retrieving Multimodal Information for Augmented Generation: A Surveycited by 84
  11. 2023Automatic Evaluation of Attribution by Large Language Modelscited by 70
  12. 2021Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One?cited by 59
38 more
  1. 2024Fine Tuning vs. Retrieval Augmented Generation for Less Popular Knowledgecited by 49
  2. 2022Questions Are All You Need to Train a Dense Passage Retrievercited by 47
  3. 2021Learning to Retrieve Passages without Supervisioncited by 46
  4. 2024A Survey of Prompt Engineering Methods in Large Language Models for Different NLP Taskscited by 46
  5. 2024How Easily do Irrelevant Inputs Skew the Responses of Large Language Models?cited by 44
  6. 2021Mention Memory: incorporating textual knowledge into Transformers through entity mention attentioncited by 42
  7. 2022QAMPARI: An Open-domain Question Answering Benchmark for Questions with Many Answers from Multiple Paragraphscited by 42
  8. 2023Give Me the Facts! A Survey on Factual Knowledge Probing in Pre-trained Language Modelscited by 32
  9. 2023WebBrain: Learning to Generate Factually Correct Articles for Queries by Grounding on Large Web Corpuscited by 29
  10. 2022What Are You Token About? Dense Retrieval as Distributions Over the Vocabularycited by 28
  11. 2023Rethinking the Role of Token Retrieval in Multi-Vector Retrievalcited by 28
  12. 2024Nearest Neighbor Speculative Decoding for LLM Generation and Attributioncited by 26
  13. 2023Decomposing Complex Queries for Tip-of-the-tongue Retrievalcited by 24
  14. 2022CharacterBERT and Self-Teaching for Improving the Robustness of Dense Retrievers on Queries with Typoscited by 22
  15. 2022Improving Biomedical Information Retrieval with Neural Retrieverscited by 20
  16. 2023Retrieval-Pretrained Transformer: Long-range Language Modeling with Self-retrievalcited by 20
  17. 2024Beneath the Surface of Consistency: Exploring Cross-lingual Knowledge Representation Sharing in LLMscited by 20
  18. 2022RoMQA: A Benchmark for Robust, Multi-evidence, Multi-answer Question Answeringcited by 19
  19. 2022CITADEL: Conditional Token Interaction via Dynamic Lexical Routing for Efficient and Effective Multi-Vector Retrievalcited by 19
  20. 2023Chain-of-Skills: A Configurable Model for Open-domain Question Answeringcited by 19
  21. 2024Retrieval Helps or Hurts? A Deeper Dive into the Efficacy of Retrieval Augmentation to Language Modelscited by 19
  22. 2024Are Large Language Models a Good Replacement of Taxonomies?cited by 19
  23. 2025Improving Code Localization with Repository Memorycited by 19
  24. 2024Narrowing the Knowledge Evaluation Gap: Open-Domain Question Answering with Multi-Granularity Answerscited by 18
  25. 2024Compressing Long Context for Enhancing RAG with AMR-based Concept Distillationcited by 16
  26. 2025Collapse of Dense Retrievers: Short, Early, and Literal Biases Outranking Factual Evidencecited by 16
  27. 2023Expand, Rerank, and Retrieve: Query Reranking for Open-Domain Question Answeringcited by 15
  28. 2024Robust Neural Information Retrieval: An Adversarial and Out-of-distribution Perspectivecited by 15
  29. 2022Empowering Language Models with Knowledge Graph Reasoning for Question Answeringcited by 14
  30. 2025Don't lie to your friends: Learning what you know from collaborative self-playcited by 14
  31. 2024RuleRAG: Rule-Guided Retrieval-Augmented Generation with Language Models for Question Answeringcited by 13
  32. 2024SMARTCAL: An Approach to Self-Aware Tool-Use Evaluation and Calibrationcited by 13
  33. 2025Rankify: A Comprehensive Python Toolkit for Retrieval, Re-Ranking, and Retrieval-Augmented Generationcited by 13
  34. 2026Thinking to Recall: How Reasoning Unlocks Parametric Knowledge in LLMscited by 13
  35. 2022Multi-Vector Retrieval as Sparse Alignmentcited by 12
  36. 2024Performance Gap in Entity Knowledge Extraction Across Modalities in Vision Language Modelscited by 12
  37. 2023The Effect of Scaling, Retrieval Augmentation and Form on the Factual Consistency of Language Modelscited by 11
  38. 2024ClaimVer: Explainable Claim-Level Verification and Evidence Attribution of Text Through Knowledge Graphscited by 11

50 most cited of 140 All 140 (JSON)