# A Wrong Answer or a Wrong Question? An Intricate Relationship between Question Reformulation and Answer Selection in Conversational Question Answering

arXiv: https://arxiv.org/abs/2010.06835 · HTML: https://mlsanity.com/paper/2010.06835/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Oct 2020.

## Built on by (most cited first, 7 of 7)

- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Reverse Question Answering: Can an LLM Write a Question so Hard (or Bad) that it Can't Answer?](https://mlsanity.com/paper/2410.15512/) (2024)
- [Robust Training for Conversational Question Answering Models with Reinforced Reformulation Generation](https://mlsanity.com/paper/2310.13505/) (2023)
- [SCAI-QReCC Shared Task on Conversational Question Answering](https://mlsanity.com/paper/2201.11094/) (2022)
- [Learning to Select the Relevant History Turns in Conversational Question Answering](https://mlsanity.com/paper/2308.02294/) (2023)
- [Keeping the Questions Conversational: Using Structured Representations to Resolve Dependency in Conversational Question Answering](https://mlsanity.com/paper/2304.07125/) (2023)
- [Leveraging Query Resolution and Reading Comprehension for Conversational Passage Retrieval](https://mlsanity.com/paper/2102.08795/) (2021)

## Abstract (arXiv)

The dependency between an adequate question formulation and correct answer selection is a very intriguing but still underexplored area. In this paper, we show that question rewriting (QR) of the conversational context allows to shed more light on this phenomenon and also use it to evaluate robustness of different answer selection approaches. We introduce a simple framework that enables an automated analysis of the conversational question answering (QA) performance using question rewrites, and present the results of this analysis on the TREC CAsT and QuAC (CANARD) datasets. Our experiments uncover sensitivity to question formulation of the popular state-of-the-art models for reading comprehension and passage ranking. Our results demonstrate that the reading comprehension model is insensitive to question formulation, while the passage ranking changes dramatically with a little variation in the input question. The benefit of QR is that it allows us to pinpoint and group such cases automatically. We show how to use this methodology to verify whether QA models are really learning the task or just finding shortcuts in the dataset, and better understand the frequent types of error they make.