# On Approximate Nearest Neighbour Selection for Multi-Stage Dense Retrieval

arXiv: https://arxiv.org/abs/2108.11480 · HTML: https://mlsanity.com/paper/2108.11480/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Aug 2021.

## Built on by (most cited first, 7 of 7)

- [ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction](https://mlsanity.com/paper/2112.01488/) (2021)
- [PLAID: An Efficient Engine for Late Interaction Retrieval](https://mlsanity.com/paper/2205.09707/) (2022)
- [Rethinking the Role of Token Retrieval in Multi-Vector Retrieval](https://mlsanity.com/paper/2304.01982/) (2023)
- [Global Minimizers of Sigmoid Contrastive Loss](https://mlsanity.com/paper/2509.18552/) (2025)
- [DisastIR: A Comprehensive Information Retrieval Benchmark for Disaster Management](https://mlsanity.com/paper/2505.15856/) (2025)
- [Leveraging Approximate Caching for Faster Retrieval-Augmented Generation](https://mlsanity.com/paper/2503.05530/) (2025)
- [Enhancing LLMs in Predictive Political QA with Semi-Structured Data](https://mlsanity.com/paper/2608.21218/) (2026)

## Abstract (arXiv)

Dense retrieval, which describes the use of contextualised language models such as BERT to identify documents from a collection by leveraging approximate nearest neighbour (ANN) techniques, has been increasing in popularity. Two families of approaches have emerged, depending on whether documents and queries are represented by single or multiple embeddings. ColBERT, the exemplar of the latter, uses an ANN index and approximate scores to identify a set of candidate documents for each query embedding, which are then re-ranked using accurate document representations. In this manner, a large number of documents can be retrieved for each query, hindering the efficiency of the approach. In this work, we investigate the use of ANN scores for ranking the candidate documents, in order to decrease the number of candidate documents being fully scored. Experiments conducted on the MSMARCO passage ranking corpus demonstrate that, by cutting of the candidate set by using the approximate scores to only 200 documents, we can still obtain an effective ranking without statistically significant differences in effectiveness, and resulting in a 2x speedup in efficiency.