# BERT-CoQAC: BERT-based Conversational Question Answering in Context

arXiv: https://arxiv.org/abs/2104.11394 · HTML: https://mlsanity.com/paper/2104.11394/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 14 references that are in the map.
- First posted on arXiv 23 Apr 2021.

## Builds on (most cited first, 12 of 12)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [CoQA: A Conversational Question Answering Challenge](https://mlsanity.com/paper/1808.07042/) (2018)
- [The NarrativeQA Reading Comprehension Challenge](https://mlsanity.com/paper/1712.07040/) (2017)
- [QuAC : Question Answering in Context](https://mlsanity.com/paper/1808.07036/) (2018)
- [Bidirectional Attention Flow for Machine Comprehension](https://mlsanity.com/paper/1611.01603/) (2016)
- [Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models](https://mlsanity.com/paper/1507.04808/) (2015)
- [BERT with History Answer Embedding for Conversational Question Answering](https://mlsanity.com/paper/1905.05412/) (2019)
- [GraphFlow: Exploiting Conversation Flow with Graph Neural Networks for Conversational Machine Comprehension](https://mlsanity.com/paper/1908.00059/) (2019)
- [Hierarchical Recurrent Attention Network for Response Generation](https://mlsanity.com/paper/1701.07149/) (2017)
- [FlowDelta: Modeling Flow Information Gain in Reasoning for Conversational Machine Comprehension](https://mlsanity.com/paper/1908.05117/) (2019)

## Built on by (most cited first, 13 of 13)

- [Securing Large Language Models: Threats, Vulnerabilities and Responsible Practices](https://mlsanity.com/paper/2403.12503/) (2024)
- [Conversational Question Answering: A Survey](https://mlsanity.com/paper/2106.00874/) (2021)
- [Decoding the AI Pen: Techniques and Challenges in Detecting AI-Generated Text](https://mlsanity.com/paper/2403.05750/) (2024)
- [A new hope for network model generalization](https://mlsanity.com/paper/2207.05843/) (2022)
- [Exploration of Masked and Causal Language Modelling for Text Generation](https://mlsanity.com/paper/2405.12630/) (2024)
- [Learning to Select the Relevant History Turns in Conversational Question Answering](https://mlsanity.com/paper/2308.02294/) (2023)
- [GRAF: Graph Retrieval Augmented by Facts for Romanian Legal Multi-Choice Question Answering](https://mlsanity.com/paper/2412.04119/) (2024)
- [Keeping the Questions Conversational: Using Structured Representations to Resolve Dependency in Conversational Question Answering](https://mlsanity.com/paper/2304.07125/) (2023)
- [Towards Adaptive Context Management for Intelligent Conversational Question Answering](https://mlsanity.com/paper/2509.17829/) (2025)
- [A Survey of the State-of-the-Art in Conversational Question Answering Systems](https://mlsanity.com/paper/2509.05716/) (2025)
- [Building a Question Answering System for the Manufacturing Domain](https://mlsanity.com/paper/2111.10044/) (2021)
- [Proactive Detractor Detection Framework Based on Message-Wise Sentiment Analysis Over Customer Support Interactions](https://mlsanity.com/paper/2211.03923/) (2022)
- [SDEC: Semantic Deep Embedded Clustering](https://mlsanity.com/paper/2508.15823/) (2025)

## Abstract (arXiv)

As one promising way to inquire about any particular information through a dialog with the bot, question answering dialog systems have gained increasing research interests recently. Designing interactive QA systems has always been a challenging task in natural language processing and used as a benchmark to evaluate a machine's ability of natural language understanding. However, such systems often struggle when the question answering is carried out in multiple turns by the users to seek more information based on what they have already learned, thus, giving rise to another complicated form called Conversational Question Answering (CQA). CQA systems are often criticized for not understanding or utilizing the previous context of the conversation when answering the questions. To address the research gap, in this paper, we explore how to integrate conversational history into the neural machine comprehension system. On one hand, we introduce a framework based on a publically available pre-trained language model called BERT for incorporating history turns into the system. On the other hand, we propose a history selection mechanism that selects the turns that are relevant and contributes the most to answer the current question. Experimentation results revealed that our framework is comparable in performance with the state-of-the-art models on the QuAC leader board. We also conduct a number of experiments to show the side effects of using entire context information which brings unnecessary information and noise signals resulting in a decline in the model's performance.