# A Simple but Effective Method to Incorporate Multi-turn Context with BERT for Conversational Machine Comprehension

arXiv: https://arxiv.org/abs/1905.12848 · HTML: https://mlsanity.com/paper/1905.12848/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 May 2019.

## Built on by (most cited first, 18 of 18)

- [Neural Approaches to Conversational Information Retrieval](https://mlsanity.com/paper/2201.05176/) (2022)
- [Conversational Question Answering: A Survey](https://mlsanity.com/paper/2106.00874/) (2021)
- [Learn to Resolve Conversational Dependency: A Consistency Training Framework for Conversational Question Answering](https://mlsanity.com/paper/2106.11575/) (2021)
- [A Short Survey of Pre-trained Language Models for Conversational AI-A NewAge in NLP](https://mlsanity.com/paper/2104.10810/) (2021)
- [ChainCQG: Flow-Aware Conversational Question Generation](https://mlsanity.com/paper/2102.02864/) (2021)
- [Ditch the Gold Standard: Re-evaluating Conversational Question Answering](https://mlsanity.com/paper/2112.08812/) (2021)
- [Accurate, yet inconsistent? Consistency Analysis on Language Understanding Models](https://mlsanity.com/paper/2108.06665/) (2021)
- [CoHS-CQG: Context and History Selection for Conversational Question Generation](https://mlsanity.com/paper/2209.06652/) (2022)
- [We've had this conversation before: A Novel Approach to Measuring Dialog Similarity](https://mlsanity.com/paper/2110.05780/) (2021)
- [Out-of-Domain Intent Detection Considering Multi-Turn Dialogue Contexts](https://mlsanity.com/paper/2305.03237/) (2023)
- [Learning to Select the Relevant History Turns in Conversational Question Answering](https://mlsanity.com/paper/2308.02294/) (2023)
- [Keeping the Questions Conversational: Using Structured Representations to Resolve Dependency in Conversational Question Answering](https://mlsanity.com/paper/2304.07125/) (2023)
- [Opinion Prediction with User Fingerprinting](https://mlsanity.com/paper/2108.00270/) (2021)
- [NoiER: An Approach for Training more Reliable Fine-TunedDownstream Task Models](https://mlsanity.com/paper/2110.02054/) (2021)
- [A Survey of the State-of-the-Art in Conversational Question Answering Systems](https://mlsanity.com/paper/2509.05716/) (2025)
- [Towards leveraging latent knowledge and Dialogue context for real-world conversational question answering](https://mlsanity.com/paper/2212.08946/) (2022)
- [QASE Enhanced PLMs: Improved Control in Text Generation for MRC](https://mlsanity.com/paper/2403.04771/) (2024)
- [Enhancing Pre-Trained Generative Language Models with Question Attended Span Extraction on Machine Reading Comprehension](https://mlsanity.com/paper/2404.17991/) (2024)

## Abstract (arXiv)

Conversational machine comprehension (CMC) requires understanding the context of multi-turn dialogue. Using BERT, a pre-training language model, has been successful for single-turn machine comprehension, while modeling multiple turns of question answering with BERT has not been established because BERT has a limit on the number and the length of input sequences. In this paper, we propose a simple but effective method with BERT for CMC. Our method uses BERT to encode a paragraph independently conditioned with each question and each answer in a multi-turn context. Then, the method predicts an answer on the basis of the paragraph representations encoded with BERT. The experiments with representative CMC datasets, QuAC and CoQA, show that our method outperformed recently published methods (+0.8 F1 on QuAC and +2.1 F1 on CoQA). In addition, we conducted a detailed analysis of the effects of the number and types of dialogue history on the accuracy of CMC, and we found that the gold answer history, which may not be given in an actual conversation, contributed to the model performance most on both datasets.