# Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback

arXiv: https://arxiv.org/abs/2108.13454 · HTML: https://mlsanity.com/paper/2108.13454/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 36 references that are in the map.
- First posted on arXiv 30 Aug 2021.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models](https://mlsanity.com/paper/2104.08663/) (2021)
- [ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT](https://mlsanity.com/paper/2004.12832/) (2020)
- [Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval](https://mlsanity.com/paper/2007.00808/) (2020)
- [Latent Retrieval for Weakly Supervised Open Domain Question Answering](https://mlsanity.com/paper/1906.00300/) (2019)
- [Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation](https://mlsanity.com/paper/2004.09813/) (2020)
- [Overview of the TREC 2019 deep learning track](https://mlsanity.com/paper/2003.07820/) (2020)
- [Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Sampling](https://mlsanity.com/paper/2104.06967/) (2021)
- [Sparse, Dense, and Attentional Representations for Text Retrieval](https://mlsanity.com/paper/2005.00181/) (2020)
- [Overview of the TREC 2020 deep learning track](https://mlsanity.com/paper/2102.07662/) (2021)
- [A Deep Relevance Matching Model for Ad-hoc Retrieval](https://mlsanity.com/paper/1711.08611/) (2017)
- [Learning Dense Representations for Entity Retrieval](https://mlsanity.com/paper/1909.10506/) (2019)
- [End-to-End Neural Ad-hoc Ranking with Kernel Pooling](https://mlsanity.com/paper/1706.06613/) (2017)
- [Learning a Deep Listwise Context Model for Ranking Refinement](https://mlsanity.com/paper/1804.05936/) (2018)
- [Distilling Dense Representations for Ranking using Tightly-Coupled Teachers](https://mlsanity.com/paper/2010.11386/) (2020)
- [Pseudo-Relevance Feedback for Multiple Representation Dense Retrieval](https://mlsanity.com/paper/2106.11251/) (2021)
- [How Deep is your Learning: the DL-HARD Annotated Deep Learning Dataset](https://mlsanity.com/paper/2105.07975/) (2021)
- [NPRF: A Neural Pseudo Relevance Feedback Framework for Ad-hoc Information Retrieval](https://mlsanity.com/paper/1810.12936/) (2018)
- [Learning To Retrieve: How to Train a Dense Retrieval Model Effectively and Efficiently](https://mlsanity.com/paper/2010.10469/) (2020)
- [Co-BERT: A Context-Aware BERT Retrieval Model Incorporating Local and Query-specific Context](https://mlsanity.com/paper/2104.08523/) (2021)
- [PGT: Pseudo Relevance Feedback Using a Graph-Based Transformer](https://mlsanity.com/paper/2101.07918/) (2021)

## Built on by (most cited first, 30 of 47)

- [TCRA-LLM: Token Compression Retrieval Augmented Large Language Model for Inference Cost Reduction](https://mlsanity.com/paper/2310.15556/) (2023)
- [RACE: Retrieval-Augmented Commit Message Generation](https://mlsanity.com/paper/2203.02700/) (2022)
- [ReGen: Zero-Shot Text Classification via Training Data Generation with Progressive Dense Retrieval](https://mlsanity.com/paper/2305.10703/) (2023)
- [GenQREnsemble: Zero-Shot LLM Ensemble Prompting for Generative Query Reformulation](https://mlsanity.com/paper/2404.03746/) (2024)
- [Utilizing BERT for Information Retrieval: Survey, Applications, Resources, and Challenges](https://mlsanity.com/paper/2403.00784/) (2024)
- [Expand, Rerank, and Retrieve: Query Reranking for Open-Domain Question Answering](https://mlsanity.com/paper/2305.17080/) (2023)
- [Taxonomy-guided Semantic Indexing for Academic Paper Search](https://mlsanity.com/paper/2410.19218/) (2024)
- [Guided Query Refinement: Multimodal Hybrid Retrieval with Test-Time Optimization](https://mlsanity.com/paper/2510.05038/) (2025)
- [Improving Retrieval in Theme-specific Applications using a Corpus Topical Taxonomy](https://mlsanity.com/paper/2403.04160/) (2024)
- [Scientific Paper Retrieval with LLM-Guided Semantic-Based Ranking](https://mlsanity.com/paper/2505.21815/) (2025)
- [Neural Rankers for Effective Screening Prioritisation in Medical Systematic Review Literature Search](https://mlsanity.com/paper/2212.09017/) (2022)
- [Optimizing Test-Time Query Representations for Dense Retrieval](https://mlsanity.com/paper/2205.12680/) (2022)
- [Generalized Contrastive Learning for Multi-Modal Retrieval and Ranking](https://mlsanity.com/paper/2404.08535/) (2024)
- [Improving Scientific Document Retrieval with Concept Coverage-based Query Set Generation](https://mlsanity.com/paper/2502.11181/) (2025)
- [Augmenting Zero-Shot Dense Retrievers with Plug-in Mixture-of-Memories](https://mlsanity.com/paper/2302.03754/) (2023)
- [DREQ: Document Re-Ranking Using Entity-based Query Understanding](https://mlsanity.com/paper/2401.05939/) (2024)
- [Generative Query Reformulation Using Ensemble Prompting, Document Fusion, and Relevance Feedback](https://mlsanity.com/paper/2405.17658/) (2024)
- [Incorporating Relevance Feedback for Information-Seeking Retrieval using Few-Shot Document Re-Ranking](https://mlsanity.com/paper/2210.10695/) (2022)
- [Relevance Feedback with Brain Signals](https://mlsanity.com/paper/2312.05669/) (2023)
- [QAEA-DR: A Unified Text Augmentation Framework for Dense Retrieval](https://mlsanity.com/paper/2407.20207/) (2024)
- [Pseudo Relevance Feedback is Enough to Close the Gap Between Small and Large Dense Retrieval Models](https://mlsanity.com/paper/2503.14887/) (2025)
- [E2Rank: Unifying Text Embedding and Listwise Reranking for Effective and Efficient Search](https://mlsanity.com/paper/2510.22733/) (2025)
- [Test-time Corpus Feedback: From Retrieval to RAG](https://mlsanity.com/paper/2508.15437/) (2025)
- [Beyond Single-Shot: Multi-step Tool Retrieval via Query Planning](https://mlsanity.com/paper/2601.07782/) (2026)
- [Asyncval: A Toolkit for Asynchronously Validating Dense Retriever Checkpoints during Training](https://mlsanity.com/paper/2202.12510/) (2022)
- [ReFIT: Relevance Feedback from a Reranker during Inference](https://mlsanity.com/paper/2305.11744/) (2023)
- [LLM-VPRF: Large Language Model Based Vector Pseudo Relevance Feedback](https://mlsanity.com/paper/2504.01448/) (2025)
- [Succeeding at Scale: Enterprise Retrieval Benchmark Construction and Index-Preserving Query Adaptation for Multi-Tenant Search](https://mlsanity.com/paper/2601.04646/) (2026)
- [LoL: A Comparative Regularization Loss over Query Reformulation Losses for Pseudo-Relevance Feedback](https://mlsanity.com/paper/2204.11545/) (2022)
- [Cross-Model Comparative Loss for Enhancing Neuronal Utility in Language Understanding](https://mlsanity.com/paper/2301.03765/) (2023)

## Abstract (arXiv)

Dense retrieval systems conduct first-stage retrieval using embedded representations and simple similarity metrics to match a query to documents. Its effectiveness depends on encoded embeddings to capture the semantics of queries and documents, a challenging task due to the shortness and ambiguity of search queries. This paper proposes ANCE-PRF, a new query encoder that uses pseudo relevance feedback (PRF) to improve query representations for dense retrieval. ANCE-PRF uses a BERT encoder that consumes the query and the top retrieved documents from a dense retrieval model, ANCE, and it learns to produce better query embeddings directly from relevance labels. It also keeps the document index unchanged to reduce overhead. ANCE-PRF significantly outperforms ANCE and other recent dense retrieval systems on several datasets. Analysis shows that the PRF encoder effectively captures the relevant and complementary information from PRF documents, while ignoring the noise with its learned attention mechanism.