ClueReader: Heterogeneous Graph Attention Network for Multi-hop Machine Reading Comprehension
Not on the map. Why?
Abstract (arXiv)
Multi-hop machine reading comprehension is a challenging task in natural language processing as it requires more reasoning ability across multiple documents. Spectral models based on graph convolutional networks have shown good inferring abilities and lead to competitive results. However, the analysis and reasoning of some are inconsistent with those of humans. Inspired by the concept of grandmother cells in cognitive neuroscience, we propose a heterogeneous graph attention network model named ClueReader to imitate the grandmother cell concept. The model is designed to assemble the semantic features in multi-level representations and automatically concentrate or alleviate information for reasoning through the attention mechanism. The name ClueReader is a metaphor for the pattern of the model: it regards the subjects of queries as the starting points of clues, takes the reasoning entities as bridge points, considers the latent candidate entities as grandmother cells, and the clues end up in candidate entities. The proposed model enables the visualization of the reasoning graph, making it possible to analyze the importance of edges connecting entities and the selectivity in the mention and candidate nodes, which is easier to comprehend empirically. Evaluations on the open-domain multi-hop reading dataset WikiHop and drug-drug interaction dataset MedHop proved the validity of ClueReader and showed the feasibility of its application of the model in the molecular biology domain.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 23 of its 44 references that are in the map. First posted on arXiv 2 Jul 2021.
Builds on
23 of 44 references.
References · 23
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
- 2017Graph Attention Networkscited by 8,854
- 2017Neural Message Passing for Quantum Chemistrycited by 3,460
- 2020Longformer: The Long-Document Transformercited by 3,362
- 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
- 2018Deep contextualized word representationscited by 2,285
- 2018Relational inductive biases, deep learning, and graph networkscited by 1,341
- 2016Bidirectional Attention Flow for Machine Comprehensioncited by 278
- 2017Constructing Datasets for Multi-hop Reading Comprehension Across Documentscited by 190
11 more
- 2019Cognitive Graph for Multi-Hop Reading Comprehension at Scalecited by 65
- 2016A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Taskscited by 62
- 2018Question Answering by Reasoning Across Documents with Graph Convolutional Networkscited by 56
- 2019Multi-hop Reading Comprehension across Multiple Documents by Reasoning over Heterogeneous Graphscited by 53
- 2016Attention-over-Attention Neural Networks for Reading Comprehensioncited by 31
- 2018Neural Models for Reasoning over Multiple Mentions using Coreferencecited by 27
- 2019BAG: Bi-directional Attention Entity Graph Convolutional Network for Multi-hop Reasoning Question Answeringcited by 19
- 2018Multi-Passage Machine Reading Comprehension with Cross-Passage Answer Verificationcited by 17
- 2020DuReader_robust: A Chinese Dataset Towards Evaluating Robustness and Generalization of Machine Reading Comprehension in Real-World Applicationscited by 13
- 2019Coarse-grain Fine-grain Coattention Network for Multi-evidence Question Answeringcited by 12
- 2020Multi-hop Reading Comprehension across Documents with Path-based Graph Convolutional Networkcited by 9
Built on by
Citing papers per year: 2022: 1.
1 paper.