# Corpus-informed Retrieval Augmented Generation of Clarifying Questions

arXiv: https://arxiv.org/abs/2409.18575 · HTML: https://mlsanity.com/paper/2409.18575/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation.
- Builds on 22 of its 47 references that are in the map.
- First posted on arXiv 27 Sep 2024.

## Placed in

- Retrieval-augmented generation / [RAG robustness and adaptive retrieval](https://mlsanity.com/rag/rag-robustness-and-adaptive-retrieval/)

## Builds on (most cited first, 22 of 22)

- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering](https://mlsanity.com/paper/2007.01282/) (2020)
- [AmbigQA: Answering Ambiguous Open-domain Questions](https://mlsanity.com/paper/2004.10645/) (2020)
- [Distilling Knowledge from Reader to Retriever for Question Answering](https://mlsanity.com/paper/2012.04584/) (2020)
- [Hurdles to Progress in Long-form Question Answering](https://mlsanity.com/paper/2103.06332/) (2021)
- [Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Information](https://mlsanity.com/paper/1805.04655/) (2018)
- [ConvAI3: Generating Clarifying Questions for Open-Domain Dialogue Systems (ClariQ)](https://mlsanity.com/paper/2009.11352/) (2020)
- [Answer-based Adversarial Training for Generating Clarification Questions](https://mlsanity.com/paper/1904.02281/) (2019)
- [Joint Passage Ranking for Diverse Multi-Answer Retrieval](https://mlsanity.com/paper/2104.08445/) (2021)
- [Analyzing and Characterizing User Intent in Information-seeking Conversations](https://mlsanity.com/paper/1804.08759/) (2018)
- [ClarQ: A large-scale and diverse dataset for Clarification Question Generation](https://mlsanity.com/paper/2006.05986/) (2020)
- [MIMICS: A Large-Scale Data Collection for Search Clarification](https://mlsanity.com/paper/2006.10174/) (2020)
- [Introducing MANtIS: a novel Multi-Domain Information Seeking Dialogues Dataset](https://mlsanity.com/paper/1912.04639/) (2019)
- [Towards Question-based Recommender Systems](https://mlsanity.com/paper/2005.14255/) (2020)
- [Zero-shot Clarifying Question Generation for Conversational Search](https://mlsanity.com/paper/2301.12660/) (2023)
- [Analysing the Effect of Clarifying Questions on Document Ranking in Conversational Search](https://mlsanity.com/paper/2008.03717/) (2020)
- [Diverse and Specific Clarification Question Generation with Keywords](https://mlsanity.com/paper/2104.10317/) (2021)
- [Bayesian Metric Learning for Uncertainty Quantification in Image Retrieval](https://mlsanity.com/paper/2302.01332/) (2023)
- [The Effects of System Initiative during Conversational Collaborative Search](https://mlsanity.com/paper/2202.09728/) (2022)
- [A Clarifying Question Selection System from NTES_ALONG in Convai3 Challenge](https://mlsanity.com/paper/2010.14202/) (2020)
- [A Comparative Study of Training Objectives for Clarification Facet Generation](https://mlsanity.com/paper/2310.00703/) (2023)

## Built on by (most cited first, 2 of 2)

- [RAC: Retrieval-Augmented Clarification for Faithful Conversational Search](https://mlsanity.com/paper/2601.11722/) (2026)
- [Sensitivity-Aware Retrieval-Augmented Intent Clarification](https://mlsanity.com/paper/2603.06025/) (2026)

## Abstract (arXiv)

This study aims to develop models that generate corpus informed clarifying questions for web search, in a way that ensures the questions align with the available information in the retrieval corpus. We demonstrate the effectiveness of Retrieval Augmented Language Models (RAG) in this process, emphasising their ability to (i) jointly model the user query and retrieval corpus to pinpoint the uncertainty and ask for clarifications end-to-end and (ii) model more evidence documents, which can be used towards increasing the breadth of the questions asked. However, we observe that in current datasets search intents are largely unsupported by the corpus, which is problematic both for training and evaluation. This causes question generation models to ``hallucinate'', ie. suggest intents that are not in the corpus, which can have detrimental effects in performance. To address this, we propose dataset augmentation methods that align the ground truth clarifications with the retrieval corpus. Additionally, we explore techniques to enhance the relevance of the evidence pool during inference, but find that identifying ground truth intents within the corpus remains challenging. Our analysis suggests that this challenge is partly due to the bias of current datasets towards clarification taxonomies and calls for data that can support generating corpus-informed clarifications.