mlsanityData to 7 Oct 2026

Greener GRASS: Enhancing GNNs with Encoding, Rewiring, and Attention

arXiv 2407.05649 ↗First posted 8 Jul 2024cs.LG cs.AI cs.NEcited by 1

Not on the map. Why?

Abstract (arXiv)

Graph Neural Networks (GNNs) have become important tools for machine learning on graph-structured data. In this paper, we explore the synergistic combination of graph encoding, graph rewiring, and graph attention, by introducing Graph Attention with Stochastic Structures (GRASS), a novel GNN architecture. GRASS utilizes relative random walk probabilities (RRWP) encoding and a novel decomposed variant (D-RRWP) to efficiently capture structural information. It rewires the input graph by superimposing a random regular graph to enhance long-range information propagation. It also employs a novel additive attention mechanism tailored for graph-structured data. Our empirical evaluations demonstrate that GRASS achieves state-of-the-art performance on multiple benchmark datasets, including a 20.3% reduction in mean absolute error on the ZINC dataset.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 43 of its 62 references that are in the map. First posted on arXiv 8 Jul 2024.

Builds on

43 of 62 references.

References · 43

  1. 2017Attention Is All You Needcited by 59,932
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  3. 2017Graph Attention Networkscited by 8,854
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  5. 2017Inductive Representation Learning on Large Graphscited by 6,284
  6. 2016Layer Normalizationcited by 5,579
  7. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  8. 2018How Powerful are Graph Neural Networks?cited by 4,355
  9. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
  10. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
  11. 2017Decoupled Weight Decay Regularizationcited by 1,722
  12. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
31 more
  1. 2022Recipe for a General, Powerful, Scalable Graph Transformercited by 702
  2. 2016Language Modeling with Gated Convolutional Networkscited by 688
  3. 2019DropEdge: Towards Deep Graph Convolutional Networks on Node Classificationcited by 679
  4. 2020Benchmarking Graph Neural Networkscited by 672
  5. 2020A Generalization of Transformer Networks to Graphscited by 653
  6. 2020On the Bottleneck of Graph Neural Networks and its Practical Implicationscited by 624
  7. 2019Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological Viewcited by 544
  8. 2021Understanding over-squashing and bottlenecks on graphs via curvaturecited by 471
  9. 2021Rethinking Graph Transformers with Spectral Attentioncited by 468
  10. 2023Symbolic Discovery of Optimization Algorithmscited by 423
  11. 2017Residual Gated Graph ConvNetscited by 341
  12. 2023A critical look at the evaluation of GNNs under heterophily: Are we really making progress?cited by 332
  13. 2021Graph Neural Networks with Learnable Structural and Positional Representationscited by 326
  14. 2022Long Range Graph Benchmarkcited by 258
  15. 2020Understanding the Difficulty of Training Transformerscited by 210
  16. 2022DeepNet: Scaling Transformers to 1,000 Layerscited by 176
  17. 2023Exphormer: Sparse Transformers for Graphscited by 166
  18. 2023Graph Inductive Biases in Transformers without Message Passingcited by 150
  19. 2021Accurate Learning of Graph Representations with Graph Multiset Poolingcited by 120
  20. 2021Global Self-Attention as a Replacement for Graph Convolutioncited by 113
  21. 2022FoSR: First-order spectral rewiring for addressing oversquashing in GNNscited by 104
  22. 2023Understanding Oversquashing in GNNs through the Lens of Effective Resistancecited by 94
  23. 2023Edge Directionality Improves Learning on Heterophilic Graphscited by 88
  24. 2023Everything is Connected: Graph Neural Networkscited by 85
  25. 2023Where Did the Gap Go? Reassessing the Long-Range Graph Benchmarkcited by 83
  26. 2023DRew: Dynamically Rewired Message Passing with Delaycited by 81
  27. 2022DiffWire: Inductive Graph Rewiring via the Lov'asz Boundcited by 79
  28. 2017Regularizing Deep Neural Networks by Noise: Its Interpretation and Optimizationcited by 56
  29. 2019DropAttention: A Regularization Method for Fully-Connected Self-Attention Networkscited by 20
  30. 2024Learning Long Range Dependencies on Graphs via Random Walkscited by 18
  31. 2022DropKeycited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Local Message-Passing for Discrete Graph Generationcited by 0