mlsanityData to 14 Oct 2026

Graph Neural Networks for Natural Language Processing: A Survey

arXiv 2106.06090 ↗First posted 10 Jun 2021cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Deep learning has become the dominant approach in coping with various tasks in Natural LanguageProcessing (NLP). Although text inputs are typically represented as a sequence of tokens, there isa rich variety of NLP problems that can be best expressed with a graph structure. As a result, thereis a surge of interests in developing new deep learning techniques on graphs for a large numberof NLP tasks. In this survey, we present a comprehensive overview onGraph Neural Networks(GNNs) for Natural Language Processing. We propose a new taxonomy of GNNs for NLP, whichsystematically organizes existing research of GNNs for NLP along three axes: graph construction,graph representation learning, and graph based encoder-decoder models. We further introducea large number of NLP applications that are exploiting the power of GNNs and summarize thecorresponding benchmark datasets, evaluation metrics, and open-source codes. Finally, we discussvarious outstanding challenges for making the full use of GNNs for NLP as well as future researchdirections. To the best of our knowledge, this is the first comprehensive overview of Graph NeuralNetworks for Natural Language Processing.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 65 of its 457 references that are in the map. First posted on arXiv 10 Jun 2021.

Builds on

65 of 457 references.

References · 65

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  5. 2016Layer Normalizationcited by 5,669
  6. 2018YOLOv3: An Incremental Improvementcited by 2,659
  7. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,434
  8. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  9. 2015A large annotated corpus for learning natural language inferencecited by 1,847
  10. 2020Linformer: Self-Attention with Linear Complexitycited by 1,474
  11. 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,026
  12. 2019Strategies for Pre-training Graph Neural Networkscited by 971
38 more
  1. 2014Embedding Entities and Relations for Learning and Inference in Knowledge Basescited by 932
  2. 2015Gated Graph Sequence Neural Networkscited by 796
  3. 2018Hierarchical Graph Representation Learning with Differentiable Poolingcited by 787
  4. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  5. 2017Convolutional 2D Knowledge Graph Embeddingscited by 730
  6. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 687
  7. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 662
  8. 2020A Survey on Knowledge Graphs: Representation, Acquisition and Applicationscited by 643
  9. 2017Representation Learning on Graphs: Methods and Applicationscited by 457
  10. 2019Multi-News: a Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Modelcited by 431
  11. 2019Composition-based Multi-Relational Graph Convolutional Networkscited by 396
  12. 2018Generating Wikipedia by Summarizing Long Sequencescited by 335
  13. 2016Deep Biaffine Attention for Neural Dependency Parsingcited by 334
  14. 2018SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inferencecited by 333
  15. 2015Deep Convolutional Networks on Graph-Structured Datacited by 246
  16. 2015Order Matters: Sequence to sequence for setscited by 237
  17. 2016Incorporating Copying Mechanism in Sequence-to-Sequence Learningcited by 218
  18. 2018ReCoRD: Bridging the Gap between Human and Machine Commonsense Reading Comprehensioncited by 200
  19. 2018code2seq: Generating Sequences from Structured Representations of Codecited by 179
  20. 2018Bottom-Up Abstractive Summarizationcited by 177
  21. 2015Large-scale Simple Question Answering with Memory Networkscited by 127
  22. 2019Fine-tune BERT for Extractive Summarizationcited by 122
  23. 2020Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question Answeringcited by 122
  24. 2016Modeling Coverage for Neural Machine Translationcited by 110
  25. 2018Event2Mind: Commonsense Inference on Events, Intents, and Reactionscited by 82
  26. 2020Improved Code Summarization via a Graph Neural Networkcited by 79
  27. 2020Every Document Owns Its Structure: Inductive Text Classification via Graph Neural Networkscited by 77
  28. 2021DIG: A Turnkey Library for Diving into Graph Deep Learning Researchcited by 76
  29. 2020Leveraging Graph to Improve Abstractive Multi-Document Summarizationcited by 67
  30. 2017Text Summarization Techniques: A Brief Surveycited by 59
  31. 2017A parallel corpus of Python functions and documentation strings for automated code documentation and code generationcited by 57
  32. 2018A Graph-to-Sequence Model for AMR-to-Text Generationcited by 56
  33. 2018Neural Document Summarization by Jointly Learning to Score and Select Sentencescited by 56
  34. 2017Neural Natural Language Inference Models Enhanced with External Knowledgecited by 40
  35. 2018Graph2Seq: Graph to Sequence Learning with Attention-based Neural Networkscited by 37
  36. 2018Abstract Meaning Representation for Multi-Document Summarizationcited by 35
  37. 2018Jointly Predicting Predicates and Arguments in Neural Semantic Role Labelingcited by 34
  38. 2018Learning to Represent Editscited by 34

50 most cited of 65 All 65 (JSON)

Built on by

Not cited yet.