mlsanityData to 7 Oct 2026

Understanding Attention and Generalization in Graph Neural Networks

arXiv 1905.02850 ↗First posted 8 May 2019cs.LG cs.AI stat.MLcited by 169

Not on the map. Why?

Abstract (arXiv)

We aim to better understand attention over nodes in graph neural networks (GNNs) and identify factors influencing its effectiveness. We particularly focus on the ability of attention GNNs to generalize to larger, more complex or noisy graphs. Motivated by insights from the work on Graph Isomorphism Networks, we design simple graph reasoning tasks that allow us to study attention in a controlled environment. We find that under typical conditions the effect of attention is negligible or even harmful, but under certain conditions it provides an exceptional gain in performance of more than 60% in some of our classification tasks. Satisfying these conditions in practice is challenging and often requires optimal initialization or supervised training of attention. We propose an alternative recipe and train attention in a weakly-supervised fashion that approaches the performance of supervised models, and, compared to unsupervised models, improves results on several synthetic as well as real datasets. Source code and datasets are available at https://github.com/bknyaz/graph_attention_pool.

Key facts. Cited by 169 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 24; 2022: 33; 2023: 50; 2024: 28; 2025: 21; 2026: 13.

169 papers.

Sort:

By area:

  1. 2022Discovering Invariant Rationales for Graph Neural Networkscited by 186
  2. 2022Interpretable and Generalizable Graph Learning via Stochastic Attention Mechanismcited by 171
  3. 2022Learning Causally Invariant Representations for Out-of-Distribution Generalization on Graphscited by 144
  4. 2021Causal Attention for Interpretable and Generalizable Graph Classificationcited by 108
  5. 2022How to Find Your Friendly Neighborhood: Graph Attention Design with Self-Supervisioncited by 102
  6. 2022Universal Prompt Tuning for Graph Neural Networkscited by 100
  7. 2022Out-Of-Distribution Generalization on Graphs: A Surveycited by 97
  8. 2022Debiasing Graph Neural Networks via Learning Disentangled Causal Substructurecited by 83
  9. 2023Attending to Graph Transformerscited by 81
  10. 2021OOD-GNN: Out-of-Distribution Generalized Graph Neural Networkcited by 78
  11. 2021Understanding Pooling in Graph Neural Networkscited by 69
  12. 2023GraphCare: Enhancing Healthcare Predictions with Personalized Knowledge Graphscited by 57
38 more
  1. 2022Graph Pooling for Graph Neural Networks: Progress, Challenges, and Opportunitiescited by 52
  2. 2021How Framelets Enhance Graph Neural Networkscited by 46
  3. 2023Does Invariant Graph Learning via Environment Augmentation Learn Invariance?cited by 42
  4. 2023Towards Self-Interpretable Graph-Level Anomaly Detectioncited by 41
  5. 2021Online Graph Dictionary Learningcited by 37
  6. 2023The expressive power of pooling in Graph Neural Networkscited by 35
  7. 2023Mind the Label Shift of Augmentation-based Graph OOD Generalizationcited by 34
  8. 2023Joint Learning of Label and Environment Causal Independence for Graph Out-of-Distribution Generalizationcited by 29
  9. 2021Recognizing Predictive Substructures with Subgraph Information Bottleneckcited by 28
  10. 2023A Survey on Explainability of Graph Neural Networkscited by 28
  11. 2022Inductive Logical Query Answering in Knowledge Graphscited by 25
  12. 2021Augmentations in Graph Contrastive Learning: Current Methodological Flaws & Towards Better Practicescited by 24
  13. 2022Graph Attention Retrospectivecited by 24
  14. 2022MESH2IR: Neural Acoustic Impulse Response Generator for Complex 3D Scenescited by 23
  15. 2021Graph coarsening: From scientific computing to machine learningcited by 21
  16. 2022Template based Graph Neural Network with Optimal Transport Distancescited by 20
  17. 2022Automated Data Augmentations for Graph Classificationcited by 19
  18. 2023Data-Centric Learning from Unlabeled Graphs with Diffusion Modelcited by 19
  19. 2023Rayleigh Quotient Graph Neural Networks for Graph-level Anomaly Detectioncited by 19
  20. 2023GraphMETRO: Mitigating Complex Graph Distribution Shifts via Mixture of Aligned Expertscited by 19
  21. 2021Bridging the Gap between Spatial and Spectral Domains: A Unified Framework for Graph Neural Networkscited by 18
  22. 2024Graph Sparsification via Mixture of Graphscited by 17
  23. 2021HW2VEC: A Graph Learning Tool for Automating Hardware Securitycited by 16
  24. 2021Graph Neural Networks: Methods, Applications, and Opportunitiescited by 16
  25. 2024A Survey of Deep Graph Learning under Distribution Shifts: from Graph Out-of-Distribution Generalization to Adaptationcited by 16
  26. 2021roadscene2vec: A Tool for Extracting and Embedding Road Scene-Graphscited by 15
  27. 2022More is Better (Mostly): On the Backdoor Attacks in Federated Graph Neural Networkscited by 15
  28. 2023MMGP: a Mesh Morphing Gaussian Process-based machine learning method for regression of physical problems under non-parameterized geometrical variabilitycited by 15
  29. 2023Towards Semi-supervised Universal Graph Classificationcited by 15
  30. 2023GraphGPT: Generative Pre-trained Graph Eulerian Transformercited by 15
  31. 2022Equivariant Hypergraph Neural Networkscited by 14
  32. 2023Semi-Supervised Graph Imbalanced Regressioncited by 14
  33. 2024How Interpretable Are Interpretable Graph Neural Networks?cited by 13
  34. 2025Towards Mechanistic Interpretability of Graph Transformers via Attention Graphscited by 12
  35. 2023Interpretable Sparsification of Brain Graphs: Better Practices and Effective Designs for Graph Neural Networkscited by 11
  36. 2021What Can Knowledge Bring to Machine Learning? -- A Survey of Low-shot Learning for Structured Datacited by 10
  37. 2022Image Classification using Graph Neural Network and Multiscale Wavelet Superpixelscited by 10
  38. 2023Listen2Scene: Interactive material-aware binaural sound propagation for reconstructed 3D scenescited by 10

50 most cited of 169 All 169 (JSON)