# Teach model to answer questions after comprehending the document

arXiv: https://arxiv.org/abs/2307.08931 · HTML: https://mlsanity.com/paper/2307.08931/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 33 references that are in the map.
- First posted on arXiv 18 Jul 2023.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [TinyBERT: Distilling BERT for Natural Language Understanding](https://mlsanity.com/paper/1909.10351/) (2019)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [RACE: Large-scale ReAding Comprehension Dataset From Examinations](https://mlsanity.com/paper/1704.04683/) (2017)
- [Bidirectional Attention Flow for Machine Comprehension](https://mlsanity.com/paper/1611.01603/) (2016)
- [Rethinking Counting and Localization in Crowds:A Purely Point-Based Framework](https://mlsanity.com/paper/2107.12746/) (2021)
- [The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations](https://mlsanity.com/paper/1511.02301/) (2015)
- [UKP-Athene: Multi-Sentence Textual Entailment for Claim Verification](https://mlsanity.com/paper/1809.01479/) (2018)
- [FlowQA: Grasping Flow in History for Conversational Machine Comprehension](https://mlsanity.com/paper/1810.06683/) (2018)
- [Read + Verify: Machine Reading Comprehension with Unanswerable Questions](https://mlsanity.com/paper/1808.05759/) (2018)
- [MCScript: A Novel Dataset for Assessing Machine Comprehension Using Script Knowledge](https://mlsanity.com/paper/1803.05223/) (2018)
- [Efficient and Robust Question Answering from Minimal Context over Documents](https://mlsanity.com/paper/1805.08092/) (2018)
- [Differentiable Feature Aggregation Search for Knowledge Distillation](https://mlsanity.com/paper/2008.00506/) (2020)
- [Hierarchical Question Answering for Long Documents](https://mlsanity.com/paper/1611.01839/) (2016)
- [Multi-Passage Machine Reading Comprehension with Cross-Passage Answer Verification](https://mlsanity.com/paper/1805.02220/) (2018)
- [ReCO: A Large Scale Chinese Reading Comprehension Dataset on Opinion](https://mlsanity.com/paper/2006.12146/) (2020)
- [Joint Answering and Explanation for Visual Commonsense Reasoning](https://mlsanity.com/paper/2202.12626/) (2022)
- [U-Net: Machine Reading Comprehension with Unanswerable Questions](https://mlsanity.com/paper/1810.06638/) (2018)

## Abstract (arXiv)

Multi-choice Machine Reading Comprehension (MRC) is a challenging extension of Natural Language Processing (NLP) that requires the ability to comprehend the semantics and logical relationships between entities in a given text. The MRC task has traditionally been viewed as a process of answering questions based on the given text. This single-stage approach has often led the network to concentrate on generating the correct answer, potentially neglecting the comprehension of the text itself. As a result, many prevalent models have faced challenges in performing well on this task when dealing with longer texts. In this paper, we propose a two-stage knowledge distillation method that teaches the model to better comprehend the document by dividing the MRC task into two separate stages. Our experimental results show that the student model, when equipped with our method, achieves significant improvements, demonstrating the effectiveness of our method.