# Feature-augmented Machine Reading Comprehension with Auxiliary Tasks

arXiv: https://arxiv.org/abs/2211.09438 · HTML: https://mlsanity.com/paper/2211.09438/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 39 references that are in the map.
- First posted on arXiv 17 Nov 2022.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces](https://mlsanity.com/paper/1805.10190/) (2018)
- [Bidirectional Attention Flow for Machine Comprehension](https://mlsanity.com/paper/1611.01603/) (2016)
- [DynaBERT: Dynamic BERT with Adaptive Width and Depth](https://mlsanity.com/paper/2004.04037/) (2020)
- [BERT for Joint Intent Classification and Slot Filling](https://mlsanity.com/paper/1902.10909/) (2019)
- [QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension](https://mlsanity.com/paper/1804.09541/) (2018)
- [A Stack-Propagation Framework with Token-Level Intent Detection for Spoken Language Understanding](https://mlsanity.com/paper/1909.02188/) (2019)
- [Masked Spectrogram Prediction For Self-Supervised Audio Pre-Training](https://mlsanity.com/paper/2204.12768/) (2022)
- [Dynamic Coattention Networks For Question Answering](https://mlsanity.com/paper/1611.01604/) (2016)
- [Machine Comprehension Using Match-LSTM and Answer Pointer](https://mlsanity.com/paper/1608.07905/) (2016)
- [Decoupled Side Information Fusion for Sequential Recommendation](https://mlsanity.com/paper/2204.11046/) (2022)
- [Stochastic Answer Networks for Machine Reading Comprehension](https://mlsanity.com/paper/1712.03556/) (2017)
- [Adaptive Bi-directional Attention: Exploring Multi-Granularity Representations for Machine Reading Comprehension](https://mlsanity.com/paper/2012.10877/) (2020)
- [Text Understanding with the Attention Sum Reader Network](https://mlsanity.com/paper/1603.01547/) (2016)
- [Equivariant Contrastive Learning for Sequential Recommendation](https://mlsanity.com/paper/2211.05290/) (2022)
- [Joint Multiple Intent Detection and Slot Filling via Self-distillation](https://mlsanity.com/paper/2108.08042/) (2021)
- [METS-CoV: A Dataset of Medical Entity and Targeted Sentiment on COVID-19 Related Tweets](https://mlsanity.com/paper/2209.13773/) (2022)
- [PIN: A Novel Parallel Interactive Network for Spoken Language Understanding](https://mlsanity.com/paper/2009.13431/) (2020)
- [Towards Joint Intent Detection and Slot Filling via Higher-order Attention](https://mlsanity.com/paper/2109.08890/) (2021)
- [A Dynamic Graph Interactive Framework with Label-Semantic Injection for Spoken Language Understanding](https://mlsanity.com/paper/2211.04023/) (2022)
- [Iterative Alternating Neural Attention for Machine Reading](https://mlsanity.com/paper/1606.02245/) (2016)
- [A Syntax-aware Multi-task Learning Framework for Chinese Semantic Role Labeling](https://mlsanity.com/paper/1911.04641/) (2019)
- [Audio-Oriented Multimodal Machine Comprehension: Task, Dataset and Model](https://mlsanity.com/paper/2107.01571/) (2021)
- [Calibrate and Refine! A Novel and Agile Framework for ASR-error Robust Intent Detection](https://mlsanity.com/paper/2205.11008/) (2022)
- [Low-resource Accent Classification in Geographically-proximate Settings: A Forensic and Sociophonetics Perspective](https://mlsanity.com/paper/2206.12759/) (2022)

## Abstract (arXiv)

While most successful approaches for machine reading comprehension rely on single training objective, it is assumed that the encoder layer can learn great representation through the loss function we define in the predict layer, which is cross entropy in most of time, in the case that we first use neural networks to encode the question and paragraph, then directly fuse the encoding result of them. However, due to the distantly loss backpropagating in reading comprehension, the encoder layer cannot learn effectively and be directly supervised. Thus, the encoder layer can not learn the representation well at any time. Base on this, we propose to inject multi granularity information to the encoding layer. Experiments demonstrate the effect of adding multi granularity information to the encoding layer can boost the performance of machine reading comprehension system. Finally, empirical study shows that our approach can be applied to many existing MRC models.