# Reinforced Question Rewriting for Conversational Question Answering

arXiv: https://arxiv.org/abs/2210.15777 · HTML: https://mlsanity.com/paper/2210.15777/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 23 references that are in the map.
- First posted on arXiv 27 Oct 2022.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [CoQA: A Conversational Question Answering Challenge](https://mlsanity.com/paper/1808.07042/) (2018)
- [Sequence Level Training with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06732/) (2015)
- [Self-critical Sequence Training for Image Captioning](https://mlsanity.com/paper/1612.00563/) (2016)
- [Deep Reinforcement Learning for Dialogue Generation](https://mlsanity.com/paper/1606.01541/) (2016)
- [Higher-order Coreference Resolution with Coarse-to-fine Inference](https://mlsanity.com/paper/1804.05392/) (2018)
- [Open-Domain Question Answering Goes Conversational via Question Rewriting](https://mlsanity.com/paper/2010.04898/) (2020)
- [Question Rewriting for Conversational Question Answering](https://mlsanity.com/paper/2004.14652/) (2020)
- [CONQRR: Conversational Query Rewriting for Retrieval with Reinforcement Learning](https://mlsanity.com/paper/2112.08558/) (2021)
- [TREC CAsT 2019: The Conversational Assistance Track Overview](https://mlsanity.com/paper/2003.13624/) (2020)
- [Ask the Right Questions: Active Question Reformulation with Reinforcement Learning](https://mlsanity.com/paper/1705.07830/) (2017)
- [Task-Oriented Query Reformulation with Reinforcement Learning](https://mlsanity.com/paper/1704.04572/) (2017)
- [Learn to Resolve Conversational Dependency: A Consistency Training Framework for Conversational Question Answering](https://mlsanity.com/paper/2106.11575/) (2021)
- [Hyperbolic Representation Learning for Fast and Efficient Neural Question Answering](https://mlsanity.com/paper/1707.07847/) (2017)

## Built on by (most cited first, 19 of 19)

- [ChatQA: Surpassing GPT-4 on Conversational QA and RAG](https://mlsanity.com/paper/2401.10225/) (2024)
- [Enhancing Conversational Search: Large Language Model-Aided Informative Query Rewriting](https://mlsanity.com/paper/2310.09716/) (2023)
- [RaFe: Ranking Feedback Improves Query Rewriting for RAG](https://mlsanity.com/paper/2405.14431/) (2024)
- [A Survey of Conversational Search](https://mlsanity.com/paper/2410.15576/) (2024)
- [Generalizing Conversational Dense Retrieval via LLM-Cognition Data Augmentation](https://mlsanity.com/paper/2402.07092/) (2024)
- [Aligning Query Representation with Rewritten Query and Relevance Judgments in Conversational Search](https://mlsanity.com/paper/2407.20189/) (2024)
- [Wizard of Shopping: Target-Oriented E-commerce Dialogue Generation with Decision Tree Branching](https://mlsanity.com/paper/2502.00969/) (2025)
- [ConvSDG: Session Data Generation for Conversational Search](https://mlsanity.com/paper/2403.11335/) (2024)
- [Robust Training for Conversational Question Answering Models with Reinforced Reformulation Generation](https://mlsanity.com/paper/2310.13505/) (2023)
- [PCoQA: Persian Conversational Question Answering Dataset](https://mlsanity.com/paper/2312.04362/) (2023)
- [Learning to Select the Relevant History Turns in Conversational Question Answering](https://mlsanity.com/paper/2308.02294/) (2023)
- [Keeping the Questions Conversational: Using Structured Representations to Resolve Dependency in Conversational Question Answering](https://mlsanity.com/paper/2304.07125/) (2023)
- [Answering Unanswered Questions through Semantic Reformulations in Spoken QA](https://mlsanity.com/paper/2305.17393/) (2023)
- [Query Enhanced Knowledge-Intensive Conversation via Unsupervised Joint Modeling](https://mlsanity.com/paper/2212.09588/) (2022)
- [Dialogue Agents 101: A Beginner's Guide to Critical Ingredients for Designing Effective Conversational Systems](https://mlsanity.com/paper/2307.07255/) (2023)
- [Controllable Decontextualization of Yes/No Question and Answers into Factual Statements](https://mlsanity.com/paper/2401.09775/) (2024)
- [Adaptive Personalized Conversational Information Retrieval](https://mlsanity.com/paper/2508.08634/) (2025)
- [A Survey of the State-of-the-Art in Conversational Question Answering Systems](https://mlsanity.com/paper/2509.05716/) (2025)
- [SEAL: Self-Evolving Agentic Learning for Conversational Question Answering over Knowledge Graphs](https://mlsanity.com/paper/2512.04868/) (2025)

## Abstract (arXiv)

Conversational Question Answering (CQA) aims to answer questions contained within dialogues, which are not easily interpretable without context. Developing a model to rewrite conversational questions into self-contained ones is an emerging solution in industry settings as it allows using existing single-turn QA systems to avoid training a CQA model from scratch. Previous work trains rewriting models using human rewrites as supervision. However, such objectives are disconnected with QA models and therefore more human-like rewrites do not guarantee better QA performance. In this paper we propose using QA feedback to supervise the rewriting model with reinforcement learning. Experiments show that our approach can effectively improve QA performance over baselines for both extractive and retrieval QA. Furthermore, human evaluation shows that our method can generate more accurate and detailed rewrites when compared to human annotations.