# Robust Training for Conversational Question Answering Models with Reinforced Reformulation Generation

arXiv: https://arxiv.org/abs/2310.13505 · HTML: https://mlsanity.com/paper/2310.13505/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 86 references that are in the map.
- First posted on arXiv 20 Oct 2023.

## Builds on (most cited first, 30 of 48)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [QuAC : Question Answering in Context](https://mlsanity.com/paper/1808.07036/) (2018)
- [Open-Domain Question Answering Goes Conversational via Question Rewriting](https://mlsanity.com/paper/2010.04898/) (2020)
- [Open-Retrieval Conversational Question Answering](https://mlsanity.com/paper/2005.11364/) (2020)
- [DisentQA: Disentangling Parametric and Contextual Knowledge with Counterfactual Question Answering](https://mlsanity.com/paper/2211.05655/) (2022)
- [Training Question Answering Models From Synthetic Data](https://mlsanity.com/paper/2002.09599/) (2020)
- [Question Rewriting for Conversational Question Answering](https://mlsanity.com/paper/2004.14652/) (2020)
- [Few-Shot Generative Conversational Query Rewriting](https://mlsanity.com/paper/2006.05009/) (2020)
- [Improving Question Answering Model Robustness with Synthetic Adversarial Data Generation](https://mlsanity.com/paper/2104.08678/) (2021)
- [Complex Sequential Question Answering: Towards Learning to Converse Over Linked Question Answer Pairs with a Knowledge Graph](https://mlsanity.com/paper/1801.10314/) (2018)
- [BERT with History Answer Embedding for Conversational Question Answering](https://mlsanity.com/paper/1905.05412/) (2019)
- [ConvGQR: Generative Query Reformulation for Conversational Search](https://mlsanity.com/paper/2305.15645/) (2023)
- [Query Resolution for Conversational Search with Limited Supervision](https://mlsanity.com/paper/2005.11723/) (2020)
- [Multi-step Retriever-Reader Interaction for Scalable Open-domain Question Answering](https://mlsanity.com/paper/1905.05733/) (2019)
- [Learning to Paraphrase for Question Answering](https://mlsanity.com/paper/1708.06022/) (2017)
- [Ask the Right Questions: Active Question Reformulation with Reinforcement Learning](https://mlsanity.com/paper/1705.07830/) (2017)
- [Rainier: Reinforced Knowledge Introspector for Commonsense Question Answering](https://mlsanity.com/paper/2210.03078/) (2022)
- [Task-Oriented Query Reformulation with Reinforcement Learning](https://mlsanity.com/paper/1704.04572/) (2017)
- [Attentive History Selection for Conversational Question Answering](https://mlsanity.com/paper/1908.09456/) (2019)
- [Multi-Task Learning for Conversational Question Answering over a Large-Scale Knowledge Base](https://mlsanity.com/paper/1910.05069/) (2019)
- [ComQA: A Community-sourced Dataset for Complex Factoid Question Answering with Paraphrase Clusters](https://mlsanity.com/paper/1809.09528/) (2018)
- [Learn to Resolve Conversational Dependency: A Consistency Training Framework for Conversational Question Answering](https://mlsanity.com/paper/2106.11575/) (2021)
- [Conversational Question Answering over Knowledge Graphs with Transformer and Graph Attention Networks](https://mlsanity.com/paper/2104.01569/) (2021)
- [Exploiting Simulated User Feedback for Conversational Search: Ranking, Rewriting, and Beyond](https://mlsanity.com/paper/2304.13874/) (2023)
- [Data Augmentation for BERT Fine-Tuning in Open-Domain Question Answering](https://mlsanity.com/paper/1904.06652/) (2019)
- [Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Space](https://mlsanity.com/paper/2010.01475/) (2020)
- [Reinforced Question Rewriting for Conversational Question Answering](https://mlsanity.com/paper/2210.15777/) (2022)

## Built on by (most cited first, 7 of 7)

- [Learning When to Retrieve, What to Rewrite, and How to Respond in Conversational QA](https://mlsanity.com/paper/2409.15515/) (2024)
- [Preference-based Learning with Retrieval Augmented Generation for Conversational Question Answering](https://mlsanity.com/paper/2503.22303/) (2025)
- [Towards Adaptive Context Management for Intelligent Conversational Question Answering](https://mlsanity.com/paper/2509.17829/) (2025)
- [A Survey of the State-of-the-Art in Conversational Question Answering Systems](https://mlsanity.com/paper/2509.05716/) (2025)
- [SEAL: Self-Evolving Agentic Learning for Conversational Question Answering over Knowledge Graphs](https://mlsanity.com/paper/2512.04868/) (2025)
- [Smart Trial: Evaluating the Use of Large Language Models for Recruiting Clinical Trial Participants via Social Media](https://mlsanity.com/paper/2509.10584/) (2025)
- [Which bird does not have wings: Negative-constrained KGQA with Schema-guided Semantic Matching and Self-directed Refinement](https://mlsanity.com/paper/2604.14749/) (2026)

## Abstract (arXiv)

Models for conversational question answering (ConvQA) over knowledge graphs (KGs) are usually trained and tested on benchmarks of gold QA pairs. This implies that training is limited to surface forms seen in the respective datasets, and evaluation is on a small set of held-out questions. Through our proposed framework REIGN, we take several steps to remedy this restricted learning setup. First, we systematically generate reformulations of training questions to increase robustness of models to surface form variations. This is a particularly challenging problem, given the incomplete nature of such questions. Second, we guide ConvQA models towards higher performance by feeding it only those reformulations that help improve their answering quality, using deep reinforcement learning. Third, we demonstrate the viability of training major model components on one benchmark and applying them zero-shot to another. Finally, for a rigorous evaluation of robustness for trained models, we use and release large numbers of diverse reformulations generated by prompting GPT for benchmark test sets (resulting in 20x increase in sizes). Our findings show that ConvQA models with robust training via reformulations, significantly outperform those with standard training from gold QA pairs only.