mlsanityData to 7 Oct 2026

How to Find Your Friendly Neighborhood: Graph Attention Design with Self-Supervision

arXiv 2204.04879 ↗First posted 11 Apr 2022cs.LG cs.AI cs.SIcited by 102

Not on the map. Why?

Abstract (arXiv)

Attention mechanism in graph neural networks is designed to assign larger weights to important neighbor nodes for better representation. However, what graph attention learns is not understood well, particularly when graphs are noisy. In this paper, we propose a self-supervised graph attention network (SuperGAT), an improved graph attention model for noisy graphs. Specifically, we exploit two attention forms compatible with a self-supervised task to predict edges, whose presence and absence contain the inherent information about the importance of the relationships between nodes. By encoding edges, SuperGAT learns more expressive attention in distinguishing mislinked neighbors. We find two graph characteristics influence the effectiveness of attention forms and self-supervision: homophily and average degree. Thus, our recipe provides guidance on which attention design to use when those two graph characteristics are known. Our experiment on 17 real-world datasets demonstrates that our recipe generalizes across 15 datasets of them, and our models designed by recipe show improved performance over baselines.

Key facts. Cited by 102 papers in the map (data through 7 Oct 2026). Builds on 49 of its 71 references that are in the map. First posted on arXiv 11 Apr 2022.

Builds on

49 of 71 references.

References · 49

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  5. 2017Graph Attention Networkscited by 8,854
  6. 2017Inductive Representation Learning on Large Graphscited by 6,284
  7. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  8. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  9. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  10. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
  11. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
  12. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
37 more
  1. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,085
  2. 2016node2vec: Scalable Feature Learning for Networkscited by 1,913
  3. 2018Graph Neural Networks: A Review of Methods and Applicationscited by 1,571
  4. 2016Variational Graph Auto-Encoderscited by 1,373
  5. 2013Spectral Networks and Locally Connected Networks on Graphscited by 1,097
  6. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,061
  7. 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,024
  8. 2018Pitfalls of Graph Neural Network Evaluationcited by 994
  9. 2016Revisiting Semi-Supervised Learning with Graph Embeddingscited by 970
  10. 2019Strategies for Pre-training Graph Neural Networkscited by 964
  11. 2018Link Prediction Based on Graph Neural Networkscited by 829
  12. 2020Geom-GCN: Geometric Graph Convolutional Networkscited by 821
  13. 2015Convolutional Networks on Graphs for Learning Molecular Fingerprintscited by 751
  14. 2015Image-based Recommendations on Styles and Substitutescited by 702
  15. 2019GraphSAINT: Graph Sampling Based Inductive Learning Methodcited by 618
  16. 2015LINE: Large-scale Information Network Embeddingcited by 582
  17. 2019Multi-scale Attributed Node Embeddingcited by 543
  18. 2019MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixingcited by 442
  19. 2019Diffusion Improves Graph Learningcited by 414
  20. 2017Deep Gaussian Embedding of Graphs: Unsupervised Inductive Learning via Rankingcited by 332
  21. 2015Diffusion-Convolutional Neural Networkscited by 268
  22. 2020Wiki-CS: A Wikipedia-Based Benchmark for Graph Neural Networkscited by 253
  23. 2015Deep Convolutional Networks on Graph-Structured Datacited by 246
  24. 2019Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networkscited by 241
  25. 2019Learning Discrete Structures for Graph Neural Networkscited by 228
  26. 2019Understanding Attention and Generalization in Graph Neural Networkscited by 169
  27. 2017Deep Learning is Robust to Massive Label Noisecited by 164
  28. 2018GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphscited by 164
  29. 2017Predicting multicellular function through multi-layer tissue networkscited by 138
  30. 2020When Does Self-Supervision Help Graph Convolutional Networks?cited by 121
  31. 2019Multi-Stage Self-Supervised Learning for Graph Convolutional Networks on Graphs with Few Labelscited by 111
  32. 2018Attention-based Graph Neural Network for Semi-supervised Learningcited by 100
  33. 2019Symmetric Graph Convolutional Autoencoder for Unsupervised Graph Representation Learningcited by 94
  34. 2018Adversarially Regularized Graph Autoencoder for Graph Embeddingcited by 84
  35. 2019Improving Graph Attention Networks with Large Margin-based Constraintscited by 34
  36. 2017Supervising Neural Attention Models for Video Captioning by Human Gaze Datacited by 18
  37. 2019Graph Representation Learning via Hard and Channel-Wise Attention Networkscited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 27; 2023: 28; 2024: 36; 2025: 5; 2026: 6.

102 papers.

Sort:

By area:

  1. 2022Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphscited by 258
  2. 2023Language is All a Graph Needscited by 219
  3. 2022A Comprehensive Survey on Trustworthy Graph Neural Networks: Privacy, Robustness, Fairness, and Explainabilitycited by 118
  4. 2022BOND: Benchmarking Unsupervised Outlier Node Detection on Static Attributed Graphscited by 80
  5. 2023Graph Prompt Learning: A Comprehensive Survey and Beyondcited by 61
  6. 2023Unnoticeable Backdoor Attacks on Graph Neural Networkscited by 45
  7. 2023Inductive Graph Unlearningcited by 30
  8. 2022CLNode: Curriculum Learning for Node Classificationcited by 24
  9. 2023Towards Deep Attention in Graph Neural Networks: Problems and Remediescited by 24
  10. 2023Counterfactual Learning on Graphs: A Surveycited by 22
  11. 2022TF-GNN: Graph Neural Networks in TensorFlowcited by 21
  12. 2022Improving Graph Neural Networks with Learnable Propagation Operatorscited by 19
38 more
  1. 2024A Survey of Data-Efficient Graph Learningcited by 18
  2. 2022Rumor Detection with Self-supervised Learning on Texts and Social Graphcited by 15
  3. 2022pathGCN: Learning General Graph Spatial Operators from Pathscited by 15
  4. 2022Towards Prototype-Based Self-Explainable Graph Neural Networkcited by 14
  5. 2024GraphFM: A generalist graph transformer that learns transferable representations across diverse domainscited by 14
  6. 2022ES-GNN: Generalizing Graph Neural Networks Beyond Homophily with Edge Splittingcited by 13
  7. 2024Masked Graph Autoencoder with Non-discrete Bandwidthscited by 11
  8. 2022Learning from the Dark: Boosting Graph Convolutional Neural Networks with Diverse Negative Samplescited by 10
  9. 2022Less is More: SlimG for Accurate, Robust, and Interpretable Graph Miningcited by 10
  10. 2022Restructuring Graph for Higher Homophily via Adaptive Spectral Clusteringcited by 9
  11. 2022Finding Diverse and Predictable Subgraphs for Graph Domain Generalizationcited by 9
  12. 2023Are GATs Out of Balance?cited by 9
  13. 2024A Survey on Self-Supervised Graph Foundation Models: Knowledge-Based Perspectivecited by 9
  14. 2022Learnable Graph Convolutional Attention Networkscited by 8
  15. 2024Self-Guided Robust Graph Structure Refinementcited by 8
  16. 2022Simplifying Node Classification on Heterophilous Graphs with Compatible Label Propagationcited by 7
  17. 2022Deformable Graph Transformercited by 7
  18. 2022Graph Convolutional Neural Networks with Diverse Negative Samples via Decomposed Determinant Point Processescited by 7
  19. 2023Heterophily-Aware Graph Attention Networkcited by 7
  20. 2023Finding Heterophilic Neighbors via Confidence-based Subgraph Matching for Semi-supervised Node Classificationcited by 7
  21. 2023Spatial Heterophily Aware Graph Neural Networkscited by 7
  22. 2024Layer-diverse Negative Sampling for Graph Neural Networkscited by 7
  23. 2024Exploring Correlations of Self-Supervised Tasks for Graphscited by 7
  24. 2022Revisiting the role of heterophily in graph representation learning: An edge classification perspectivecited by 6
  25. 2023Multi-channel learning for integrating structural hierarchies into context-dependent molecular representationcited by 6
  26. 2024Learning to Approximate Adaptive Kernel Convolution on Graphscited by 6
  27. 2023Causal-Based Supervision of Attention in Graph Neural Network: A Better and Simpler Choice towards Powerful Attentioncited by 5
  28. 2023Homological Neural Networks: A Sparse Architecture for Multivariate Complexitycited by 5
  29. 2023Examining the Effects of Degree Distribution and Homophily in Graph Learning Modelscited by 5
  30. 2023SignGT: Signed Attention-based Graph Transformer for Graph Representation Learningcited by 5
  31. 2024BanglaAutoKG: Automatic Bangla Knowledge Graph Construction with Semantic Neural Graph Filteringcited by 5
  32. 2024Faithful and Accurate Self-Attention Attribution for Message Passing Neural Networks via the Computation Tree Viewpointcited by 5
  33. 2024NTFormer: A Composite Node Tokenized Graph Transformer for Node Classificationcited by 5
  34. 2024Community-Invariant Graph Contrastive Learningcited by 4
  35. 2024GATE: How to Keep Out Intrusive Neighborscited by 4
  36. 2024Global-Local Graph Neural Networks for Node-Classificationcited by 4
  37. 2022Spiking GATs: Learning Graph Attentions via Spiking Neural Networkcited by 3
  38. 2022Causally-guided Regularization of Graph Attention Improves Generalizabilitycited by 3

50 most cited of 102 All 102 (JSON)