# BERT with History Answer Embedding for Conversational Question Answering

arXiv: https://arxiv.org/abs/1905.05412 · HTML: https://mlsanity.com/paper/1905.05412/

## Key facts

- Cited by 67 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 May 2019.

## Built on by (most cited first, 30 of 67)

- [Sequence Parallelism: Long Sequence Training from System Perspective](https://mlsanity.com/paper/2105.13120/) (2021)
- [TopiOCQA: Open-domain Conversational Question Answering with Topic Switching](https://mlsanity.com/paper/2110.00768/) (2021)
- [Let the LLMs Talk: Simulating Human-to-Human Conversational QA via Zero-Shot LLM-to-LLM Interactions](https://mlsanity.com/paper/2312.02913/) (2023)
- [Go Wider Instead of Deeper](https://mlsanity.com/paper/2107.11817/) (2021)
- [Natural Language Understanding with Privacy-Preserving BERT](https://mlsanity.com/paper/2104.07504/) (2021)
- [PACIFIC: Towards Proactive Conversational Question Answering over Tabular and Textual Data in Finance](https://mlsanity.com/paper/2210.08817/) (2022)
- [MatSci-NLP: Evaluating Scientific Language Models on Materials Science Language Tasks Using Text-to-Schema Modeling](https://mlsanity.com/paper/2305.08264/) (2023)
- [Conversational Question Answering: A Survey](https://mlsanity.com/paper/2106.00874/) (2021)
- [Learn to Resolve Conversational Dependency: A Consistency Training Framework for Conversational Question Answering](https://mlsanity.com/paper/2106.11575/) (2021)
- [An Efficient DP-SGD Mechanism for Large Scale NLP Models](https://mlsanity.com/paper/2107.14586/) (2021)
- [Conversational Question Answering on Heterogeneous Sources](https://mlsanity.com/paper/2204.11677/) (2022)
- [Chart Question Answering: State of the Art and Future Directions](https://mlsanity.com/paper/2205.03966/) (2022)
- [A Short Survey of Pre-trained Language Models for Conversational AI-A NewAge in NLP](https://mlsanity.com/paper/2104.10810/) (2021)
- [Personalized Large Language Model Assistant with Evolving Conditional Memory](https://mlsanity.com/paper/2312.17257/) (2023)
- [iAgent: LLM Agent as a Shield between User and Recommender Systems](https://mlsanity.com/paper/2502.14662/) (2025)
- [Structural Pre-training for Dialogue Comprehension](https://mlsanity.com/paper/2105.10956/) (2021)
- [Ditch the Gold Standard: Re-evaluating Conversational Question Answering](https://mlsanity.com/paper/2112.08812/) (2021)
- [Saving Dense Retriever from Shortcut Dependency in Conversational Search](https://mlsanity.com/paper/2202.07280/) (2022)
- [Reinforcement Learning from Reformulations in Conversational Question Answering over Knowledge Graphs](https://mlsanity.com/paper/2105.04850/) (2021)
- [Question Answering Survey: Directions, Challenges, Datasets, Evaluation Matrices](https://mlsanity.com/paper/2112.03572/) (2021)
- [Generating Information-Seeking Conversations from Unlabeled Documents](https://mlsanity.com/paper/2205.12609/) (2022)
- [RoR: Read-over-Read for Long Document Machine Reading Comprehension](https://mlsanity.com/paper/2109.04780/) (2021)
- [BERT-CoQAC: BERT-based Conversational Question Answering in Context](https://mlsanity.com/paper/2104.11394/) (2021)
- [Enhanced Speaker-aware Multi-party Multi-turn Dialogue Comprehension](https://mlsanity.com/paper/2109.04066/) (2021)
- [Implementing BERT and fine-tuned RobertA to detect AI generated news by ChatGPT](https://mlsanity.com/paper/2306.07401/) (2023)
- [Advances in Multi-turn Dialogue Comprehension: A Survey](https://mlsanity.com/paper/2103.03125/) (2021)
- [On the Robustness of Dialogue History Representation in Conversational Question Answering: A Comprehensive Study and a New Prompt-based Method](https://mlsanity.com/paper/2206.14796/) (2022)
- [Speech Recognition by Simply Fine-tuning BERT](https://mlsanity.com/paper/2102.00291/) (2021)
- [DialogueBERT: A Self-Supervised Learning based Dialogue Pre-training Encoder](https://mlsanity.com/paper/2109.10480/) (2021)
- [Legal Question-Answering in the Indian Context: Efficacy, Challenges, and Potential of Modern AI Models](https://mlsanity.com/paper/2309.14735/) (2023)

## Abstract (arXiv)

Conversational search is an emerging topic in the information retrieval community. One of the major challenges to multi-turn conversational search is to model the conversation history to answer the current question. Existing methods either prepend history turns to the current question or use complicated attention mechanisms to model the history. We propose a conceptually simple yet highly effective approach referred to as history answer embedding. It enables seamless integration of conversation history into a conversational question answering (ConvQA) model built on BERT (Bidirectional Encoder Representations from Transformers). We first explain our view that ConvQA is a simplified but concrete setting of conversational search, and then we provide a general framework to solve ConvQA. We further demonstrate the effectiveness of our approach under this framework. Finally, we analyze the impact of different numbers of history turns under different settings to provide new insights into conversation history modeling in ConvQA.