Data to 7 Oct 2026

Sparse, Dense, and Attentional Representations for Text Retrieval

arXiv 2005.00181 ↗First posted 1 May 2020cs.CLcited by 213

Not on the map. Why?

Cited by 21 in Retrieval-augmented generation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Dual encoders perform retrieval by encoding documents and queries into dense lowdimensional vectors, scoring each document by its inner product with the query. We investigate the capacity of this architecture relative to sparse bag-of-words models and attentional neural networks. Using both theoretical and empirical analysis, we establish connections between the encoding dimension, the margin between gold and lower-ranked documents, and the document length, suggesting limitations in the capacity of fixed-length encodings to support precise retrieval of long documents. Building on these insights, we propose a simple neural model that combines the efficiency of dual encoders with some of the expressiveness of more costly attentional architectures, and explore sparse-dense hybrids to capitalize on the precision of sparse retrieval. These models outperform strong alternatives in large-scale retrieval.

Key facts. Cited by 213 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 43; 2022: 58; 2023: 33; 2024: 37; 2025: 18; 2026: 24.

213 papers.

Sort:

By area:

2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,050
2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 464
2021Large Dual Encoders Are Generalizable Retrieverscited by 454
2023Augmented Language Models: a Surveycited by 398
2021Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Samplingcited by 238
2022Promptagator: Few-shot Dense Retrieval From 8 Examplescited by 233
2021Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrievalcited by 203
2024RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMscited by 200
2021RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-rankingcited by 175
2021Condenser: a Pre-training Architecture for Dense Retrievalcited by 162

38 more

2021GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graphcited by 143
2021End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answeringcited by 133
2021COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretrainingcited by 126
2022RetroMAE: Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encodercited by 118
2021COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted Listcited by 113
2024From Matching to Generation: A Survey on Generative Information Retrievalcited by 89
2023Dense X Retrieval: What Retrieval Granularity Should We Use?cited by 88
2023Retrieving Multimodal Information for Augmented Generation: A Surveycited by 84
2021Adversarial Retriever-Ranker for dense text retrievalcited by 82
2025ReasonIR: Training Retrievers for Reasoning Taskscited by 76
2021CONQRR: Conversational Query Rewriting for Retrieval with Reinforcement Learningcited by 68
2024Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wiselycited by 65
2021The Web Is Your Oyster - Knowledge-Intensive NLP against a Very Large Web Corpuscited by 60
2021Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One?cited by 59
2021PAIR: Leveraging Passage-Centric Similarity Relation for Improving Dense Passage Retrievalcited by 50
2022Questions Are All You Need to Train a Dense Passage Retrievercited by 47
2021Learning to Retrieve Passages without Supervisioncited by 46
2021Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedbackcited by 45
2021Multi-task Retrieval for Knowledge-Intensive Taskscited by 44
2024How Easily do Irrelevant Inputs Skew the Responses of Large Language Models?cited by 44
2021Joint Passage Ranking for Diverse Multi-Answer Retrievalcited by 43
2022Neural Approaches to Conversational Information Retrievalcited by 41
2022Multi-View Document Representation Learning for Open-Domain Dense Retrievalcited by 41
2022ERNIE-Search: Bridging Cross-Encoder with Dual-Encoder via Self On-the-fly Distillation for Dense Passage Retrievalcited by 41
2021Understanding Hard Negatives in Noise Contrastive Estimationcited by 39
2022Curriculum Learning for Dense Retrieval Distillationcited by 38
2021R2-D2: A Modular Baseline for Open-Domain Question Answeringcited by 37
2022Towards Robust Ranker for Text Retrievalcited by 37
2022Order-Disorder: Imitation Adversarial Attacks for Black-box Neural Ranking Modelscited by 37
2021Multi-Vector Models with Textual Guidance for Fine-Grained Scientific Document Similaritycited by 35
2021Phrase Retrieval Learns Passage Retrieval, Toocited by 33
2022NGAME: Negative Mining-aware Mini-batching for Extreme Classificationcited by 32
2021Challenges in Generalization in Open Domain Question Answeringcited by 30
2023Rethinking the Role of Token Retrieval in Multi-Vector Retrievalcited by 28
2024Scaling Laws For Dense Retrievalcited by 28
2021Zero-Shot Dense Retrieval with Momentum Adversarial Domain Invariant Representationscited by 27
2022Multi-CPR: A Multi Domain Chinese Dataset for Passage Retrievalcited by 26
2022CORE: A Retrieve-then-Edit Framework for Counterfactual Data Generationcited by 23

50 most cited of 213 All 213 (JSON)