# On the Robustness of Dialogue History Representation in Conversational Question Answering: A Comprehensive Study and a New Prompt-based Method

arXiv: https://arxiv.org/abs/2206.14796 · HTML: https://mlsanity.com/paper/2206.14796/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 43 references that are in the map.
- First posted on arXiv 29 Jun 2022.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Big Bird: Transformers for Longer Sequences](https://mlsanity.com/paper/2007.14062/) (2020)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [CoQA: A Conversational Question Answering Challenge](https://mlsanity.com/paper/1808.07042/) (2018)
- [QuAC : Question Answering in Context](https://mlsanity.com/paper/1808.07036/) (2018)
- [How Many Data Points is a Prompt Worth?](https://mlsanity.com/paper/2103.08493/) (2021)
- [Open-Domain Question Answering Goes Conversational via Question Rewriting](https://mlsanity.com/paper/2010.04898/) (2020)
- [Open-Retrieval Conversational Question Answering](https://mlsanity.com/paper/2005.11364/) (2020)
- [TopiOCQA: Open-domain Conversational Question Answering with Topic Switching](https://mlsanity.com/paper/2110.00768/) (2021)
- [Question Rewriting for Conversational Question Answering](https://mlsanity.com/paper/2004.14652/) (2020)
- [BERT with History Answer Embedding for Conversational Question Answering](https://mlsanity.com/paper/1905.05412/) (2019)
- [PADA: Example-based Prompt Learning for on-the-fly Adaptation to Unseen Domains](https://mlsanity.com/paper/2102.12206/) (2021)
- [HTLM: Hyper-Text Pre-Training and Prompting of Language Models](https://mlsanity.com/paper/2107.06955/) (2021)
- [Conversational Question Reformulation via Sequence-to-Sequence Architectures and Pretrained Language Models](https://mlsanity.com/paper/2004.01909/) (2020)
- [DoQA -- Accessing Domain-Specific FAQs via Conversational QA](https://mlsanity.com/paper/2005.01328/) (2020)
- [GraphFlow: Exploiting Conversation Flow with Graph Neural Networks for Conversational Machine Comprehension](https://mlsanity.com/paper/1908.00059/) (2019)
- [FlowQA: Grasping Flow in History for Conversational Machine Comprehension](https://mlsanity.com/paper/1810.06683/) (2018)
- [Attentive History Selection for Conversational Question Answering](https://mlsanity.com/paper/1908.09456/) (2019)
- [SDNet: Contextualized Attention-based Deep Network for Conversational Question Answering](https://mlsanity.com/paper/1812.03593/) (2018)
- [Learn to Resolve Conversational Dependency: A Consistency Training Framework for Conversational Question Answering](https://mlsanity.com/paper/2106.11575/) (2021)
- [FlowDelta: Modeling Flow Information Gain in Reasoning for Conversational Machine Comprehension](https://mlsanity.com/paper/1908.05117/) (2019)
- [Ditch the Gold Standard: Re-evaluating Conversational Question Answering](https://mlsanity.com/paper/2112.08812/) (2021)

## Built on by (most cited first, 12 of 12)

- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agents](https://mlsanity.com/paper/2412.03563/) (2024)
- [Boosting Conversational Question Answering with Fine-Grained Retrieval-Augmentation and Self-Check](https://mlsanity.com/paper/2403.18243/) (2024)
- [Measuring the Robustness of NLP Models to Domain Shifts](https://mlsanity.com/paper/2306.00168/) (2023)
- [Robust Training for Conversational Question Answering Models with Reinforced Reformulation Generation](https://mlsanity.com/paper/2310.13505/) (2023)
- [q2d: Turning Questions into Dialogs to Teach Models How to Search](https://mlsanity.com/paper/2304.14318/) (2023)
- [Self-Supervised Position Debiasing for Large Language Models](https://mlsanity.com/paper/2401.01218/) (2024)
- [Cancer-Answer: Empowering Cancer Care with Advanced Large Language Models](https://mlsanity.com/paper/2411.06946/) (2024)
- [When Compression Becomes an Attack Surface: Black-Box Attacks on Prompt-Compressed LLM Agents](https://mlsanity.com/paper/2510.22963/) (2025)
- [Scalable LLM-based Coding of Dialogue in Healthcare Simulation: Balancing Coding Performance, Processing Time, and Environmental Impact](https://mlsanity.com/paper/2604.23255/) (2026)
- [Safe to Check, Unsafe to Use: Relinking at the Compression Boundary of LLM Agents](https://mlsanity.com/paper/2606.21732/) (2026)
- [Prompt Sensitivity of Generative Agents: Evidence from an Epidemic Model](https://mlsanity.com/paper/2608.26221/) (2026)

## Abstract (arXiv)

Most works on modeling the conversation history in Conversational Question Answering (CQA) report a single main result on a common CQA benchmark. While existing models show impressive results on CQA leaderboards, it remains unclear whether they are robust to shifts in setting (sometimes to more realistic ones), training data size (e.g. from large to small sets) and domain. In this work, we design and conduct the first large-scale robustness study of history modeling approaches for CQA. We find that high benchmark scores do not necessarily translate to strong robustness, and that various methods can perform extremely differently under different settings. Equipped with the insights from our study, we design a novel prompt-based history modeling approach, and demonstrate its strong robustness across various settings. Our approach is inspired by existing methods that highlight historic answers in the passage. However, instead of highlighting by modifying the passage token embeddings, we add textual prompts directly in the passage text. Our approach is simple, easy-to-plug into practically any model, and highly effective, thus we recommend it as a starting point for future model developers. We also hope that our study and insights will raise awareness to the importance of robustness-focused evaluation, in addition to obtaining high leaderboard scores, leading to better CQA systems.