mlsanityData to 14 Oct 2026

Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One?

arXiv 2110.06918 ↗First posted 13 Oct 2021cs.CL cs.IR cs.LGcited by 60

Not on the map. Why?

Abstract (arXiv)

Despite their recent popularity and well-known advantages, dense retrievers still lag behind sparse methods such as BM25 in their ability to reliably match salient phrases and rare entities in the query and to generalize to out-of-domain data. It has been argued that this is an inherent limitation of dense models. We rebut this claim by introducing the Salient Phrase Aware Retriever (SPAR), a dense retriever with the lexical matching capacity of a sparse model. We show that a dense Lexical Model Lambda can be trained to imitate a sparse one, and SPAR is built by augmenting a standard dense retriever with Lambda. Empirically, SPAR shows superior performance on a range of tasks including five question answering datasets, MS MARCO passage retrieval, as well as the EntityQuestions and BEIR benchmarks for out-of-domain evaluation, exceeding the performance of state-of-the-art dense and sparse retrievers. The code and models of SPAR are available at: https://github.com/facebookresearch/dpr-scale/tree/main/spar

Key facts. Cited by 60 papers in the map (data through 14 Oct 2026). Builds on 26 of its 44 references that are in the map. First posted on arXiv 13 Oct 2021.

Builds on

26 of 44 references.

References · 26

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  3. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  4. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  5. 2017Billion-scale similarity search with GPUscited by 2,497
  6. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  7. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
  8. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
  9. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 849
  10. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 695
  11. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 413
  12. 2021Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Samplingcited by 238
14 more
  1. 2020Sparse, Dense, and Attentional Representations for Text Retrievalcited by 214
  2. 2021PAQ: 65 Million Probably-Asked Questions and What You Can Do With Themcited by 200
  3. 2020Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillationcited by 145
  4. 2021Simple Entity-Centric Questions Challenge Dense Retrieverscited by 140
  5. 2020Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasetscited by 138
  6. 2021COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted Listcited by 113
  7. 2021A Few Brief Notes on DeepImpact, COIL, and a Conceptual Framework for Information Retrieval Techniquescited by 69
  8. 2021Learning Passage Impacts for Inverted Indexescited by 68
  9. 2021Domain-matched Pre-training Tasks for Dense Retrievalcited by 53
  10. 2020The Curse of Dense Low-Dimensional Information Retrieval for Large Index Sizescited by 48
  11. 2021Multi-task Retrieval for Knowledge-Intensive Taskscited by 44
  12. 2021A Replication Study of Dense Passage Retrievercited by 37
  13. 2021Challenges in Generalization in Open Domain Question Answeringcited by 30
  14. 2019Efficient Inner Product Approximation in Hybrid Spacescited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 3; 2022: 20; 2023: 14; 2024: 6; 2025: 9; 2026: 8.

60 papers.

Sort:

By area:

  1. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
  2. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
  3. 2022Promptagator: Few-shot Dense Retrieval From 8 Examplescited by 235
  4. 2022Improving Passage Retrieval with Zero-Shot Question Generationcited by 165
  5. 2024MM-Embed: Universal Multimodal Retrieval with Multimodal LLMscited by 161
  6. 2023How to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrievalcited by 118
  7. 2022Task-aware Retrieval with Instructionscited by 107
  8. 2023Dense X Retrieval: What Retrieval Granularity Should We Use?cited by 88
  9. 2021The Web Is Your Oyster - Knowledge-Intensive NLP against a Very Large Web Corpuscited by 60
  10. 2022Towards Robust Ranker for Text Retrievalcited by 37
  11. 2023Rethinking the Role of Token Retrieval in Multi-Vector Retrievalcited by 31
  12. 2022What Are You Token About? Dense Retrieval as Distributions Over the Vocabularycited by 28
38 more
  1. 2023When do Generative Query and Document Expansions Fail? A Comprehensive Study Across Methods, Retrievers, and Datasetscited by 26
  2. 2022CharacterBERT and Self-Teaching for Improving the Robustness of Dense Retrievers on Queries with Typoscited by 22
  3. 2022A Thorough Examination on Zero-shot Dense Retrievalcited by 21
  4. 2022UnifieR: A Unified Retriever for Large-Scale Retrievalcited by 19
  5. 2022CITADEL: Conditional Token Interaction via Dynamic Lexical Routing for Efficient and Effective Multi-Vector Retrievalcited by 19
  6. 2023Chain-of-Skills: A Configurable Model for Open-domain Question Answeringcited by 19
  7. 2025DRAMA: Diverse Augmentation from Large Language Models to Smaller Dense Retrieverscited by 19
  8. 2022PMC-Patients: A Large-scale Dataset of Patient Summaries and Relations for Benchmarking Retrieval-based Clinical Decision Support Systemscited by 17
  9. 2022DuReader_retrieval: A Large-scale Chinese Benchmark for Passage Retrieval from Web Search Enginecited by 15
  10. 2022LED: Lexicon-Enlightened Dense Retriever for Large-Scale Retrievalcited by 15
  11. 2023Expand, Rerank, and Retrieve: Query Reranking for Open-Domain Question Answeringcited by 15
  12. 2022Low-Resource Dense Retrieval for Open-Domain Question Answering: A Comprehensive Surveycited by 14
  13. 2025Rankify: A Comprehensive Python Toolkit for Retrieval, Re-Ranking, and Retrieval-Augmented Generationcited by 13
  14. 2022Cross-document Event Coreference Search: Task, Dataset and Modelingcited by 12
  15. 2021Boosted Dense Retrievercited by 10
  16. 2024Entity Retrieval for Answering Entity-Centric Questionscited by 10
  17. 2025CG-RAG: Research Question Answering by Citation Graph Retrieval-Augmented LLMscited by 8
  18. 2022A Study on the Efficiency and Generalization of Light Hybrid Retrieverscited by 7
  19. 2022Entity-Focused Dense Passage Retrieval for Outside-Knowledge Visual Question Answeringcited by 7
  20. 2022Zero-Shot Retrieval with Search Agents and Hybrid Environmentscited by 6
  21. 2023CAME: Competitively Learning a Mixture-of-Experts Model for First-stage Retrievalcited by 6
  22. 2023Boot and Switch: Alternating Distillation for Zero-Shot Dense Retrievalcited by 4
  23. 2024Multi-Field Adaptive Retrievalcited by 4
  24. 2025BioMol-MQA: A Multi-Modal Question Answering Dataset For LLM Reasoning Over Bio-Molecular Interactionscited by 4
  25. 2026Taxonomy of the Retrieval System Framework: Pitfalls and Paradigmscited by 4
  26. 2022Task-Aware Specialization for Efficient and Robust Dense Retrieval for Open-Domain Question Answeringcited by 3
  27. 2022AugTriever: Unsupervised Dense Retrieval and Domain Adaptation by Scalable Data Augmentationcited by 3
  28. 2023Teaching Smaller Language Models To Generalise To Unseen Compositional Questionscited by 2
  29. 2024Return of EM: Entity-driven Answer Set Expansion for QA Evaluationcited by 2
  30. 2025QuOTE: Question-Oriented Text Embeddingscited by 2
  31. 2025Fine-Grained Knowledge Structuring and Retrieval for Visual Question Answeringcited by 2
  32. 2025Dense Retrievers Can Fail on Simple Queries: Revealing The Granularity Dilemma of Embeddingscited by 2
  33. 2026Facts as First Class Objects: Knowledge Objects for Persistent LLM Memorycited by 2
  34. 2023Improving Out-of-Distribution Generalization of Neural Rerankers with Contextualized Late Interactioncited by 1
  35. 2023Unsupervised Dense Retrieval Training with Web Anchorscited by 1
  36. 2023Can a Frozen Pretrained Language Model be used for Zero-shot Neural Retrieval on Entity-centric Questions?cited by 0
  37. 2023Causal schema induction for knowledge discoverycited by 0
  38. 2023Generate to Understand for Representationcited by 0

50 most cited of 60 All 60 (JSON)