# Pseudo Relevance Feedback with Deep Language Models and Dense Retrievers: Successes and Pitfalls

arXiv: https://arxiv.org/abs/2108.11044 · HTML: https://mlsanity.com/paper/2108.11044/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Aug 2021.

## Built on by (most cited first, 26 of 26)

- [UniMS-RAG: A Unified Multi-source Retrieval-Augmented Generation for Personalized Dialogue Systems](https://mlsanity.com/paper/2401.13256/) (2024)
- [GenQREnsemble: Zero-Shot LLM Ensemble Prompting for Generative Query Reformulation](https://mlsanity.com/paper/2404.03746/) (2024)
- [LED: Lexicon-Enlightened Dense Retriever for Large-Scale Retrieval](https://mlsanity.com/paper/2208.13661/) (2022)
- [KiRAG: Knowledge-Driven Iterative Retriever for Enhancing Retrieval-Augmented Generation](https://mlsanity.com/paper/2502.18397/) (2025)
- [Can Query Expansion Improve Generalization of Strong Cross-Encoder Rankers?](https://mlsanity.com/paper/2311.09175/) (2023)
- [Neural Rankers for Effective Screening Prioritisation in Medical Systematic Review Literature Search](https://mlsanity.com/paper/2212.09017/) (2022)
- [Developing Retrieval Augmented Generation (RAG) based LLM Systems from PDFs: An Experience Report](https://mlsanity.com/paper/2410.15944/) (2024)
- [Optimizing Test-Time Query Representations for Dense Retrieval](https://mlsanity.com/paper/2205.12680/) (2022)
- [Generative Query Reformulation Using Ensemble Prompting, Document Fusion, and Relevance Feedback](https://mlsanity.com/paper/2405.17658/) (2024)
- [Constructing and Evaluating Declarative RAG Pipelines in PyTerrier](https://mlsanity.com/paper/2506.10802/) (2025)
- [Simple Yet Effective Neural Ranking and Reranking Baselines for Cross-Lingual Information Retrieval](https://mlsanity.com/paper/2304.01019/) (2023)
- [QAEA-DR: A Unified Text Augmentation Framework for Dense Retrieval](https://mlsanity.com/paper/2407.20207/) (2024)
- [E2Rank: Unifying Text Embedding and Listwise Reranking for Effective and Efficient Search](https://mlsanity.com/paper/2510.22733/) (2025)
- [A Little More Like This: Text-to-Image Retrieval with Vision-Language Models Using Relevance Feedback](https://mlsanity.com/paper/2511.17255/) (2025)
- [Beyond Single-Shot: Multi-step Tool Retrieval via Query Planning](https://mlsanity.com/paper/2601.07782/) (2026)
- [Asyncval: A Toolkit for Asynchronously Validating Dense Retriever Checkpoints during Training](https://mlsanity.com/paper/2202.12510/) (2022)
- [LLM-VPRF: Large Language Model Based Vector Pseudo Relevance Feedback](https://mlsanity.com/paper/2504.01448/) (2025)
- [LoL: A Comparative Regularization Loss over Query Reformulation Losses for Pseudo-Relevance Feedback](https://mlsanity.com/paper/2204.11545/) (2022)
- [Relevance feedback strategies for recall-oriented neural information retrieval](https://mlsanity.com/paper/2311.15110/) (2023)
- [A Representation Sharpening Framework for Zero Shot Dense Retrieval](https://mlsanity.com/paper/2511.05684/) (2025)
- [Can QPP Choose the Right Query Variant? Evaluating Query Variant Selection for RAG Pipelines](https://mlsanity.com/paper/2604.22661/) (2026)
- [Test-Time Training for Zero-Resource Dense Retrieval Reranking](https://mlsanity.com/paper/2606.01070/) (2026)
- [PLAID-PRF: Pseudo-Relevance Feedback with Centroid-like Tokens in PLAID](https://mlsanity.com/paper/2607.18626/) (2026)
- [The Laws of Context Allocation: Causal Measurement and Closed-Loop Orchestration in Generative Search](https://mlsanity.com/paper/2608.23252/) (2026)
- [Robustness of IR Models to Collection Growth](https://mlsanity.com/paper/2608.23419/) (2026)
- [A Matryoshka Hierarchical RAG for Efficient Multi-Hop Question Answering](https://mlsanity.com/paper/2610.01767/) (2026)

## Abstract (arXiv)

Pseudo Relevance Feedback (PRF) is known to improve the effectiveness of bag-of-words retrievers. At the same time, deep language models have been shown to outperform traditional bag-of-words rerankers. However, it is unclear how to integrate PRF directly with emergent deep language models. In this article, we address this gap by investigating methods for integrating PRF signals into rerankers and dense retrievers based on deep language models. We consider text-based and vector-based PRF approaches, and investigate different ways of combining and scoring relevance signals. An extensive empirical evaluation was conducted across four different datasets and two task settings (retrieval and ranking). Text-based PRF results show that the use of PRF had a mixed effect on deep rerankers across different datasets. We found that the best effectiveness was achieved when (i) directly concatenating each PRF passage with the query, searching with the new set of queries, and then aggregating the scores; (ii) using Borda to aggregate scores from PRF runs. Vector-based PRF results show that the use of PRF enhanced the effectiveness of deep rerankers and dense retrievers over several evaluation metrics. We found that higher effectiveness was achieved when (i) the query retains either the majority or the same weight within the PRF mechanism, and (ii) a shallower PRF signal (i.e., a smaller number of top-ranked passages) was employed, rather than a deeper signal. Our vector-based PRF method is computationally efficient; thus this represents a general PRF method others can use with deep rerankers and dense retrievers.