# PGT: Pseudo Relevance Feedback Using a Graph-Based Transformer

arXiv: https://arxiv.org/abs/2101.07918 · HTML: https://mlsanity.com/paper/2101.07918/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 26 references that are in the map.
- First posted on arXiv 20 Jan 2021.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [Generating Long Sequences with Sparse Transformers](https://mlsanity.com/paper/1904.10509/) (2019)
- [Latent Retrieval for Weakly Supervised Open Domain Question Answering](https://mlsanity.com/paper/1906.00300/) (2019)
- [Passage Re-ranking with BERT](https://mlsanity.com/paper/1901.04085/) (2019)
- [Complementing Lexical Retrieval with Semantic Residual Embedding](https://mlsanity.com/paper/2004.13969/) (2020)
- [Guided Transformer: Leveraging Multiple External Sources for Representation Learning in Conversational Search](https://mlsanity.com/paper/2006.07548/) (2020)
- [NPRF: A Neural Pseudo Relevance Feedback Framework for Ad-hoc Information Retrieval](https://mlsanity.com/paper/1810.12936/) (2018)
- [Relevance Transformer: Generating Concise Code Snippets with Relevance Feedback](https://mlsanity.com/paper/2007.02609/) (2020)

## Built on by (most cited first, 3 of 3)

- [Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback](https://mlsanity.com/paper/2108.13454/) (2021)
- [LLM-VPRF: Large Language Model Based Vector Pseudo Relevance Feedback](https://mlsanity.com/paper/2504.01448/) (2025)
- [Retrieval Augmentation for T5 Re-ranker using External Sources](https://mlsanity.com/paper/2210.05145/) (2022)

## Abstract (arXiv)

Most research on pseudo relevance feedback (PRF) has been done in vector space and probabilistic retrieval models. This paper shows that Transformer-based rerankers can also benefit from the extra context that PRF provides. It presents PGT, a graph-based Transformer that sparsifies attention between graph nodes to enable PRF while avoiding the high computational complexity of most Transformer architectures. Experiments show that PGT improves upon non-PRF Transformer reranker, and it is at least as accurate as Transformer PRF models that use full attention, but with lower computational costs.