mlsanityData to 7 Oct 2026

GATE: How to Keep Out Intrusive Neighbors

arXiv 2406.00418 ↗First posted 1 Jun 2024cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Graph Attention Networks (GATs) are designed to provide flexible neighborhood aggregation that assigns weights to neighbors according to their importance. In practice, however, GATs are often unable to switch off task-irrelevant neighborhood aggregation, as we show experimentally and analytically. To address this challenge, we propose GATE, a GAT extension that holds three major advantages: i) It alleviates over-smoothing by addressing its root cause of unnecessary neighborhood aggregation. ii) Similarly to perceptrons, it benefits from higher depth as it can still utilize additional layers for (non-)linear feature transformations in case of (nearly) switched-off neighborhood aggregation. iii) By down-weighting connections to unrelated neighbors, it often outperforms GATs on real-world heterophilic datasets. To further validate our claims, we construct a synthetic test bed to analyze a model's ability to utilize the appropriate amount of neighborhood aggregation, which could be of independent interest.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 40 of its 55 references that are in the map. First posted on arXiv 1 Jun 2024.

Builds on

40 of 55 references.

References · 40

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  2. 2017Graph Attention Networkscited by 8,854
  3. 2017Inductive Representation Learning on Large Graphscited by 6,284
  4. 2018How Powerful are Graph Neural Networks?cited by 4,355
  5. 2018Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learningcited by 1,099
  6. 2018Predict then Propagate: Graph Neural Networks meet Personalized PageRankcited by 987
  7. 2018Representation Learning on Graphs with Jumping Knowledge Networkscited by 875
  8. 2020Simple and Deep Graph Convolutional Networkscited by 857
  9. 2020Geom-GCN: Geometric Graph Convolutional Networkscited by 821
  10. 2019Graph WaveNet for Deep Spatial-Temporal Graph Modelingcited by 820
  11. 2021How Attentive are Graph Attention Networks?cited by 818
  12. 2019DropEdge: Towards Deep Graph Convolutional Networks on Node Classificationcited by 679
28 more
  1. 2020On the Bottleneck of Graph Neural Networks and its Practical Implicationscited by 624
  2. 2021Beyond Low-frequency Information in Graph Convolutional Networkscited by 361
  3. 2023A critical look at the evaluation of GNNs under heterophily: Are we really making progress?cited by 332
  4. 2019PairNorm: Tackling Oversmoothing in GNNscited by 297
  5. 2020Towards Deeper Graph Neural Networkscited by 296
  6. 2020A Note on Over-Smoothing for Graph Neural Networkscited by 245
  7. 2016Molecular Graph Convolutions: Moving Beyond Fingerprintscited by 238
  8. 2019Learning Discrete Structures for Graph Neural Networkscited by 228
  9. 2021Is Homophily a Necessity for Graph Neural Networks?cited by 182
  10. 2019Layer-Dependent Importance Sampling for Training Deep and Large Graph Convolutional Networkscited by 177
  11. 2020Rumor Detection on Social Media with Bi-Directional Graph Convolutional Networkscited by 152
  12. 2021DropGNN: Random Dropouts Increase the Expressiveness of Graph Neural Networkscited by 117
  13. 2020GraphNorm: A Principled Approach to Accelerating Graph Neural Network Trainingcited by 116
  14. 2020Towards Deeper Graph Neural Networks with Differentiable Group Normalizationcited by 113
  15. 2017Knowledge Transfer for Out-of-Knowledge-Base Entities: A Graph Neural Network Approachcited by 102
  16. 2022How to Find Your Friendly Neighborhood: Graph Attention Design with Self-Supervisioncited by 102
  17. 2019Inductive Matrix Completion Based on Graph Neural Networkscited by 85
  18. 2022Expander Graph Propagationcited by 75
  19. 2021On Provable Benefits of Depth in Training Graph Convolutional Networkscited by 58
  20. 2020Understanding and Resolving Performance Degradation in Graph Convolutional Networkscited by 53
  21. 2021Node Dependent Local Smoothing for Scalable Graph Learningcited by 45
  22. 2020Rethinking Graph Regularization for Graph Neural Networkscited by 36
  23. 2019Improving Graph Attention Networks with Large Margin-based Constraintscited by 34
  24. 2022Graph Attention Retrospectivecited by 24
  25. 2023Towards Deep Attention in Graph Neural Networks: Problems and Remediescited by 24
  26. 2023Adaptive Message Passing: A General Framework to Mitigate Oversmoothing, Oversquashing, and Underreachingcited by 23
  27. 2022Improving Graph Neural Networks with Learnable Propagation Operatorscited by 19
  28. 2023Are GATs Out of Balance?cited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025GNNs Getting ComFy: Community and Feature Similarity Guided Rewiringcited by 13
  2. 2025Uncertainty Estimation for Heterophilic Graphs Through the Lens of Information Theorycited by 2
  3. 2026Fixed Aggregation Features Can Rival GNNscited by 2
  4. 2026Multi-Agent Systems are Mixtures of Experts: Who Becomes an Influencer?cited by 0