mlsanityData to 7 Oct 2026

Parameter-Efficient Neural Question Answering Models via Graph-Enriched Document Representations

arXiv 2106.00851 ↗First posted 1 Jun 2021cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

As the computational footprint of modern NLP systems grows, it becomes increasingly important to arrive at more efficient models. We show that by employing graph convolutional document representation, we can arrive at a question answering system that performs comparably to, and in some cases exceeds the SOTA solutions, while using less than 5% of their resources in terms of trainable parameters. As it currently stands, a major issue in applying GCNs to NLP is document representation. In this paper, we show that a GCN enriched document representation greatly improves the results seen in HotPotQA, even when using a trivial topology. Our model (gQA), performs admirably when compared to the current SOTA, and requires little to no preprocessing. In Shao et al. 2020, the authors suggest that graph networks are not necessary for good performance in multi-hop QA. In this paper, we suggest that large language models are not necessary for good performance by showing a na"ive implementation of a GCN performs comparably to SoTA models based on pretrained language models.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 27 of its 31 references that are in the map. First posted on arXiv 1 Jun 2021.

Builds on

27 of 31 references.

References · 27

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  7. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  8. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,085
  9. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  10. 2013Spectral Networks and Locally Connected Networks on Graphscited by 1,097
  11. 2015Gated Graph Sequence Neural Networkscited by 792
  12. 2015Convolutional Networks on Graphs for Learning Molecular Fingerprintscited by 751
15 more
  1. 2014Distributed Representations of Sentences and Documentscited by 677
  2. 2015Skip-Thought Vectorscited by 321
  3. 2015Deep Convolutional Networks on Graph-Structured Datacited by 246
  4. 2018An efficient framework for learning sentence representationscited by 197
  5. 2017Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Featurescited by 103
  6. 2016Learning Distributed Representations of Sentences from Unlabelled Datacited by 90
  7. 2019Hierarchical Graph Network for Multi-hop Question Answeringcited by 83
  8. 2019Dynamically Fused Graph Network for Multi-hop Reasoningcited by 63
  9. 2018GraphIE: A Graph-Based Framework for Information Extractioncited by 45
  10. 2019Answering while Summarizing: Multi-task Learning for Multi-hop QA with Evidence Extractioncited by 39
  11. 2017Efficient Vector Representation for Documents through Corruptioncited by 16
  12. 2016Siamese CBOW: Optimizing Word Embeddings for Sentence Representationscited by 10
  13. 2019Multi-Paragraph Reasoning with Knowledge-enhanced Graph Neural Networkcited by 6
  14. 2020Is Graph Structure Necessary for Multi-hop Question Answering?cited by 6
  15. 2017Unsupervised Document Embedding With CNNscited by 1

Built on by

Not cited yet.