# Hindsight: Posterior-guided training of retrievers for improved open-ended generation

arXiv: https://arxiv.org/abs/2110.07752 · HTML: https://mlsanity.com/paper/2110.07752/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 47 references that are in the map.
- First posted on arXiv 14 Oct 2021.

## Builds on (most cited first, 29 of 29)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering](https://mlsanity.com/paper/2007.01282/) (2020)
- [ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT](https://mlsanity.com/paper/2004.12832/) (2020)
- [Generalization through Memorization: Nearest Neighbor Language Models](https://mlsanity.com/paper/1911.00172/) (2019)
- [Plug and Play Language Models: A Simple Approach to Controlled Text Generation](https://mlsanity.com/paper/1912.02164/) (2019)
- [Reading Wikipedia to Answer Open-Domain Questions](https://mlsanity.com/paper/1704.00051/) (2017)
- [Latent Retrieval for Weakly Supervised Open Domain Question Answering](https://mlsanity.com/paper/1906.00300/) (2019)
- [Retrieval Augmentation Reduces Hallucination in Conversation](https://mlsanity.com/paper/2104.07567/) (2021)
- [KILT: a Benchmark for Knowledge Intensive Language Tasks](https://mlsanity.com/paper/2009.02252/) (2020)
- [Wizard of Wikipedia: Knowledge-Powered Conversational agents](https://mlsanity.com/paper/1811.01241/) (2018)
- [Zero-Shot Relation Extraction via Reading Comprehension](https://mlsanity.com/paper/1706.04115/) (2017)
- [A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Stories](https://mlsanity.com/paper/1604.01696/) (2016)
- [Generating Wikipedia by Summarizing Long Sequences](https://mlsanity.com/paper/1801.10198/) (2018)
- [Distilling Knowledge from Reader to Retriever for Question Answering](https://mlsanity.com/paper/2012.04584/) (2020)
- [A Dataset for Document Grounded Conversations](https://mlsanity.com/paper/1809.07358/) (2018)
- [Sequential Latent Knowledge Selection for Knowledge-Grounded Dialogue](https://mlsanity.com/paper/2002.07510/) (2020)
- [Learning to Select Knowledge for Response Generation in Dialog Systems](https://mlsanity.com/paper/1902.04911/) (2019)
- [End-to-End Training of Neural Retrievers for Open-Domain Question Answering](https://mlsanity.com/paper/2101.00408/) (2021)
- [Relevance-guided Supervision for OpenQA with ColBERT](https://mlsanity.com/paper/2007.00814/) (2020)
- [Neural Generation Meets Real People: Towards Emotionally Engaging Mixed-Initiative Conversations](https://mlsanity.com/paper/2008.12348/) (2020)
- [Shallow pooling for sparse labels](https://mlsanity.com/paper/2109.00062/) (2021)

## Built on by (most cited first, 30 of 38)

- [Atlas: Few-shot Learning with Retrieval Augmented Language Models](https://mlsanity.com/paper/2208.03299/) (2022)
- [ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction](https://mlsanity.com/paper/2112.01488/) (2021)
- [Memory-Based Model Editing at Scale](https://mlsanity.com/paper/2206.06520/) (2022)
- [Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLP](https://mlsanity.com/paper/2212.14024/) (2022)
- [A Survey on Retrieval-Augmented Text Generation](https://mlsanity.com/paper/2202.01110/) (2022)
- [Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answering](https://mlsanity.com/paper/2307.16877/) (2023)
- [Autoregressive Search Engines: Generating Substrings as Document Identifiers](https://mlsanity.com/paper/2204.10628/) (2022)
- [Re2G: Retrieve, Rerank, Generate](https://mlsanity.com/paper/2207.06300/) (2022)
- [PLAID: An Efficient Engine for Late Interaction Retrieval](https://mlsanity.com/paper/2205.09707/) (2022)
- [FiD-Light: Efficient and Effective Retrieval-Augmented Text Generation](https://mlsanity.com/paper/2209.14290/) (2022)
- [The Web Is Your Oyster - Knowledge-Intensive NLP against a Very Large Web Corpus](https://mlsanity.com/paper/2112.09924/) (2021)
- [Evidentiality-guided Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2112.08688/) (2021)
- [An Efficient Memory-Augmented Transformer for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2210.16773/) (2022)
- [Stochastic RAG: End-to-End Retrieval-Augmented Generation through Expected Utility Maximization](https://mlsanity.com/paper/2405.02816/) (2024)
- [Personalized Large Language Model Assistant with Evolving Conditional Memory](https://mlsanity.com/paper/2312.17257/) (2023)
- [RichRAG: Crafting Rich Responses for Multi-faceted Queries in Retrieval-Augmented Generation](https://mlsanity.com/paper/2406.12566/) (2024)
- [ATLANTIC: Structure-Aware Retrieval-Augmented Language Model for Interdisciplinary Science](https://mlsanity.com/paper/2311.12289/) (2023)
- [GripRank: Bridging the Gap between Retrieval and Generation via the Generative Knowledge Improved Passage Ranking](https://mlsanity.com/paper/2305.18144/) (2023)
- [MobileRAG: A Fast, Memory-Efficient, and Energy-Efficient Method for On-Device RAG](https://mlsanity.com/paper/2507.01079/) (2025)
- [Multi-Task Retrieval-Augmented Text Generation with Relevance Sampling](https://mlsanity.com/paper/2207.03030/) (2022)
- [You Truly Understand What I Need: Intellectual and Friendly Dialogue Agents grounding Knowledge and Persona](https://mlsanity.com/paper/2301.02401/) (2023)
- [Passage-Mask: A Learnable Regularization Strategy for Retriever-Reader Models](https://mlsanity.com/paper/2211.00915/) (2022)
- [Variational Open-Domain Question Answering](https://mlsanity.com/paper/2210.06345/) (2022)
- [Policy-Gradient Training of Language Models for Ranking](https://mlsanity.com/paper/2310.04407/) (2023)
- [Variational Learning for Unsupervised Knowledge Grounded Dialogs](https://mlsanity.com/paper/2112.00653/) (2021)
- [KGI: An Integrated Framework for Knowledge Intensive Language Tasks](https://mlsanity.com/paper/2204.03985/) (2022)
- [Thrust: Adaptively Propels Large Language Models with External Knowledge](https://mlsanity.com/paper/2307.10442/) (2023)
- [A Statistical Framework for Data-dependent Retrieval-Augmented Models](https://mlsanity.com/paper/2408.15399/) (2024)
- [There Is No Standard Answer: Knowledge-Grounded Dialogue Generation with Adversarial Activated Multi-Reference Learning](https://mlsanity.com/paper/2210.12459/) (2022)
- [Query Enhanced Knowledge-Intensive Conversation via Unsupervised Joint Modeling](https://mlsanity.com/paper/2212.09588/) (2022)

## Abstract (arXiv)

Many text generation systems benefit from using a retriever to retrieve passages from a textual knowledge corpus (e.g., Wikipedia) which are then provided as additional context to the generator. For open-ended generation tasks (like generating informative utterances in conversations) many varied passages may be equally relevant and we find that existing methods that jointly train the retriever and generator underperform: the retriever may not find relevant passages even amongst the top-10 and hence the generator may not learn a preference to ground its generated output in them. We propose using an additional guide retriever that is allowed to use the target output and "in hindsight" retrieve relevant passages during training. We model the guide retriever after the posterior distribution Q of passages given the input and the target output and train it jointly with the standard retriever and the generator by maximizing the evidence lower bound (ELBo) in expectation over Q. For informative conversations from the Wizard of Wikipedia dataset, with posterior-guided training, the retriever finds passages with higher relevance in the top-10 (23% relative improvement), the generator's responses are more grounded in the retrieved passage (19% relative improvement) and the end-to-end system produces better overall output (6.4% relative improvement).