# ClueReader: Heterogeneous Graph Attention Network for Multi-hop Machine Reading Comprehension

arXiv: https://arxiv.org/abs/2107.00841 · HTML: https://mlsanity.com/paper/2107.00841/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 44 references that are in the map.
- First posted on arXiv 2 Jul 2021.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [Bidirectional Attention Flow for Machine Comprehension](https://mlsanity.com/paper/1611.01603/) (2016)
- [Constructing Datasets for Multi-hop Reading Comprehension Across Documents](https://mlsanity.com/paper/1710.06481/) (2017)
- [Cognitive Graph for Multi-Hop Reading Comprehension at Scale](https://mlsanity.com/paper/1905.05460/) (2019)
- [A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks](https://mlsanity.com/paper/1611.01587/) (2016)
- [Question Answering by Reasoning Across Documents with Graph Convolutional Networks](https://mlsanity.com/paper/1808.09920/) (2018)
- [Multi-hop Reading Comprehension across Multiple Documents by Reasoning over Heterogeneous Graphs](https://mlsanity.com/paper/1905.07374/) (2019)
- [Attention-over-Attention Neural Networks for Reading Comprehension](https://mlsanity.com/paper/1607.04423/) (2016)
- [Neural Models for Reasoning over Multiple Mentions using Coreference](https://mlsanity.com/paper/1804.05922/) (2018)
- [BAG: Bi-directional Attention Entity Graph Convolutional Network for Multi-hop Reasoning Question Answering](https://mlsanity.com/paper/1904.04969/) (2019)
- [Multi-Passage Machine Reading Comprehension with Cross-Passage Answer Verification](https://mlsanity.com/paper/1805.02220/) (2018)
- [DuReader_robust: A Chinese Dataset Towards Evaluating Robustness and Generalization of Machine Reading Comprehension in Real-World Applications](https://mlsanity.com/paper/2004.11142/) (2020)
- [Coarse-grain Fine-grain Coattention Network for Multi-evidence Question Answering](https://mlsanity.com/paper/1901.00603/) (2019)
- [Multi-hop Reading Comprehension across Documents with Path-based Graph Convolutional Network](https://mlsanity.com/paper/2006.06478/) (2020)

## Built on by (most cited first, 1 of 1)

- [Medical Knowledge Graph QA for Drug-Drug Interaction Prediction based on Multi-hop Machine Reading Comprehension](https://mlsanity.com/paper/2212.09400/) (2022)

## Abstract (arXiv)

Multi-hop machine reading comprehension is a challenging task in natural language processing as it requires more reasoning ability across multiple documents. Spectral models based on graph convolutional networks have shown good inferring abilities and lead to competitive results. However, the analysis and reasoning of some are inconsistent with those of humans. Inspired by the concept of grandmother cells in cognitive neuroscience, we propose a heterogeneous graph attention network model named ClueReader to imitate the grandmother cell concept. The model is designed to assemble the semantic features in multi-level representations and automatically concentrate or alleviate information for reasoning through the attention mechanism. The name ClueReader is a metaphor for the pattern of the model: it regards the subjects of queries as the starting points of clues, takes the reasoning entities as bridge points, considers the latent candidate entities as grandmother cells, and the clues end up in candidate entities. The proposed model enables the visualization of the reasoning graph, making it possible to analyze the importance of edges connecting entities and the selectivity in the mention and candidate nodes, which is easier to comprehend empirically. Evaluations on the open-domain multi-hop reading dataset WikiHop and drug-drug interaction dataset MedHop proved the validity of ClueReader and showed the feasibility of its application of the model in the molecular biology domain.