# Modular Approach to Machine Reading Comprehension: Mixture of Task-Aware Experts

arXiv: https://arxiv.org/abs/2210.01750 · HTML: https://mlsanity.com/paper/2210.01750/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 20 references that are in the map.
- First posted on arXiv 4 Oct 2022.

## Builds on (most cited first, 14 of 14)

- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [Memory Networks](https://mlsanity.com/paper/1410.3916/) (2014)
- [Machine Comprehension Using Match-LSTM and Answer Pointer](https://mlsanity.com/paper/1608.07905/) (2016)
- [AmazonQA: A Review-Based Question Answering Task](https://mlsanity.com/paper/1908.04364/) (2019)
- [Attention-over-Attention Neural Networks for Reading Comprehension](https://mlsanity.com/paper/1607.04423/) (2016)
- [ReasoNet: Learning to Stop Reading in Machine Comprehension](https://mlsanity.com/paper/1609.05284/) (2016)
- [DCN+: Mixed Objective and Deep Residual Coattention for Question Answering](https://mlsanity.com/paper/1711.00106/) (2017)
- [Yuanfudao at SemEval-2018 Task 11: Three-way Attention and Relational Knowledge for Commonsense Machine Comprehension](https://mlsanity.com/paper/1803.00191/) (2018)
- [LSTM-based Mixture-of-Experts for Knowledge-Aware Dialogues](https://mlsanity.com/paper/1605.01652/) (2016)
- [S-Net: From Answer Extraction to Answer Generation for Machine Reading Comprehension](https://mlsanity.com/paper/1706.04815/) (2017)
- [Dynamic Fusion Networks for Machine Reading Comprehension](https://mlsanity.com/paper/1711.04964/) (2017)
- [Tartan: A retrieval-based socialbot powered by a dynamic finite-state machine architecture](https://mlsanity.com/paper/1812.01260/) (2018)
- [Natural Language Comprehension with the EpiReader](https://mlsanity.com/paper/1606.02270/) (2016)

## Abstract (arXiv)

In this work we present a Mixture of Task-Aware Experts Network for Machine Reading Comprehension on a relatively small dataset. We particularly focus on the issue of common-sense learning, enforcing the common ground knowledge by specifically training different expert networks to capture different kinds of relationships between each passage, question and choice triplet. Moreover, we take inspi ration on the recent advancements of multitask and transfer learning by training each network a relevant focused task. By making the mixture-of-networks aware of a specific goal by enforcing a task and a relationship, we achieve state-of-the-art results and reduce over-fitting.