Attention-based graph neural networks: a survey
Not on the map. Why?
Abstract (arXiv)
Graph neural networks (GNNs) aim to learn well-trained representations in a lower-dimension space for downstream tasks while preserving the topological structures. In recent years, attention mechanism, which is brilliant in the fields of natural language processing and computer vision, is introduced to GNNs to adaptively select the discriminative features and automatically filter the noisy information. To the best of our knowledge, due to the fast-paced advances in this domain, a systematic overview of attention-based GNNs is still missing. To fill this gap, this paper aims to provide a comprehensive survey on recent advances in attention-based GNNs. Firstly, we propose a novel two-level taxonomy for attention-based GNNs from the perspective of development history and architectural perspectives. Specifically, the upper level reveals the three developmental stages of attention-based GNNs, including graph recurrent attention networks, graph attention networks, and graph transformers. The lower level focuses on various typical architectures of each stage. Secondly, we review these attention-based methods following the proposed taxonomy in detail and summarize the advantages and disadvantages of various models. A model characteristics table is also provided for a more comprehensive comparison. Thirdly, we share our thoughts on some open issues and future directions of attention-based GNNs. We hope this survey will provide researchers with an up-to-date reference regarding applications of attention-based GNNs. In addition, to cope with the rapid development in this field, we intend to share the relevant latest papers as an open resource at https://github.com/sunxiaobei/awesome-attention-based-gnns.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 137 of its 185 references that are in the map. First posted on arXiv 9 May 2026.
Builds on
137 of 185 references.
References · 137
- 2017Attention Is All You Needcited by 59,932
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
- 2017Graph Attention Networkscited by 8,854
- 2017Inductive Representation Learning on Large Graphscited by 6,284
- 2018How Powerful are Graph Neural Networks?cited by 4,355
- 2017Neural Message Passing for Quantum Chemistrycited by 3,460
- 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
- 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
- 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,085
- 2016node2vec: Scalable Feature Learning for Networkscited by 1,913
- 2018Graph Neural Networks: A Review of Methods and Applicationscited by 1,571
- 2014DeepWalk: Online Learning of Social Representationscited by 1,543
38 more
- 2019Simplifying Graph Convolutional Networkscited by 1,470
- 2018Relational inductive biases, deep learning, and graph networkscited by 1,341
- 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,199
- 2018Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learningcited by 1,099
- 2013Spectral Networks and Locally Connected Networks on Graphscited by 1,097
- 2018Predict then Propagate: Graph Neural Networks meet Personalized PageRankcited by 987
- 2018Representation Learning on Graphs with Jumping Knowledge Networkscited by 875
- 2021How Attentive are Graph Attention Networks?cited by 818
- 2015Gated Graph Sequence Neural Networkscited by 792
- 2018Hierarchical Graph Representation Learning with Differentiable Poolingcited by 786
- 2020Efficient Transformers: A Surveycited by 784
- 2019Heterogeneous Graph Attention Networkcited by 732
- 2019DropEdge: Towards Deep Graph Convolutional Networks on Node Classificationcited by 679
- 2020A Generalization of Transformer Networks to Graphscited by 653
- 2020On the Bottleneck of Graph Neural Networks and its Practical Implicationscited by 624
- 2019GraphSAINT: Graph Sampling Based Inductive Learning Methodcited by 618
- 2015LINE: Large-scale Information Network Embeddingcited by 582
- 2020Heterogeneous Graph Transformercited by 581
- 2019Graph Transformer Networkscited by 520
- 2020Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classificationcited by 500
- 2020Graph Neural Networks in Recommender Systems: A Surveycited by 493
- 2022Transformers in Time Series: A Surveycited by 489
- 2021Rethinking Graph Transformers with Spectral Attentioncited by 468
- 2019GMAN: A Graph Multi-Attention Network for Traffic Predictioncited by 433
- 2019Self-Attention Graph Poolingcited by 421
- 2019Diffusion Improves Graph Learningcited by 414
- 2019Hyperbolic Graph Convolutional Neural Networkscited by 400
- 2021A Survey of Transformerscited by 399
- 2019Provably Powerful Graph Networkscited by 385
- 2020Explainability in Graph Neural Networks: A Taxonomic Surveycited by 383
- 2021Beyond Low-frequency Information in Graph Convolutional Networkscited by 361
- 2019KGAT: Knowledge Graph Attention Network for Recommendationcited by 342
- 2020Towards Deeper Graph Neural Networkscited by 296
- 2021On Explainability of Graph Neural Networks via Subgraph Explorationscited by 280
- 2019Representation Learning for Dynamic Graphs: A Surveycited by 260
- 2021Graph Neural Network for Traffic Forecasting: A Surveycited by 260
- 2021Multi-Time Attention Networks for Irregularly Sampled Time Seriescited by 234
- 2021Attention Mechanisms in Computer Vision: A Surveycited by 232
50 most cited of 137 All 137 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.