# How does Feedback Signal Quality Impact Effectiveness of Pseudo Relevance Feedback for Passage Retrieval?

arXiv: https://arxiv.org/abs/2205.05888 · HTML: https://mlsanity.com/paper/2205.05888/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 May 2022.

## Built on by (most cited first, 5 of 5)

- [Neural Rankers for Effective Screening Prioritisation in Medical Systematic Review Literature Search](https://mlsanity.com/paper/2212.09017/) (2022)
- [Relevance Feedback with Brain Signals](https://mlsanity.com/paper/2312.05669/) (2023)
- [Pseudo Relevance Feedback is Enough to Close the Gap Between Small and Large Dense Retrieval Models](https://mlsanity.com/paper/2503.14887/) (2025)
- [ReFIT: Relevance Feedback from a Reranker during Inference](https://mlsanity.com/paper/2305.11744/) (2023)
- [Intended Target Identification for Anomia Patients with Gradient-based Selective Augmentation](https://mlsanity.com/paper/2506.14203/) (2025)

## Abstract (arXiv)

Pseudo-Relevance Feedback (PRF) assumes that the top results retrieved by a first-stage ranker are relevant to the original query and uses them to improve the query representation for a second round of retrieval. This assumption however is often not correct: some or even all of the feedback documents may be irrelevant. Indeed, the effectiveness of PRF methods may well depend on the quality of the feedback signal and thus on the effectiveness of the first-stage ranker. This aspect however has received little attention before. In this paper we control the quality of the feedback signal and measure its impact on a range of PRF methods, including traditional bag-of-words methods (Rocchio), and dense vector-based methods (learnt and not learnt). Our results show the important role the quality of the feedback signal plays on the effectiveness of PRF methods. Importantly, and surprisingly, our analysis reveals that not all PRF methods are the same when dealing with feedback signals of varying quality. These findings are critical to gain a better understanding of the PRF methods and of which and when they should be used, depending on the feedback signal quality, and set the basis for future research in this area.