mlsanityData to 7 Oct 2026

Gradient Gating for Deep Multi-Rate Learning on Graphs

arXiv 2210.00513 ↗First posted 2 Oct 2022cs.LG stat.MLcited by 49

Not on the map. Why?

Abstract (arXiv)

We present Gradient Gating (G2), a novel framework for improving the performance of Graph Neural Networks (GNNs). Our framework is based on gating the output of GNN layers with a mechanism for multi-rate flow of message passing information across nodes of the underlying graph. Local gradients are harnessed to further modulate message passing updates. Our framework flexibly allows one to use any basic GNN layer as a wrapper around which the multi-rate gradient gating mechanism is built. We rigorously prove that G2 alleviates the oversmoothing problem and allows the design of deep GNNs. Empirical results are presented to demonstrate that the proposed framework achieves state-of-the-art performance on a variety of graph learning tasks, including on large-scale heterophilic graphs.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). Builds on 43 of its 64 references that are in the map. First posted on arXiv 2 Oct 2022.

Builds on

43 of 64 references.

References · 43

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  2. 2017Graph Attention Networkscited by 8,854
  3. 2017Inductive Representation Learning on Large Graphscited by 6,284
  4. 2018How Powerful are Graph Neural Networks?cited by 4,355
  5. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  6. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  7. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,085
  8. 2018Graph Neural Networks: A Review of Methods and Applicationscited by 1,571
  9. 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,133
  10. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
  11. 2013Spectral Networks and Locally Connected Networks on Graphscited by 1,097
  12. 2018Predict then Propagate: Graph Neural Networks meet Personalized PageRankcited by 987
31 more
  1. 2020Simple and Deep Graph Convolutional Networkscited by 857
  2. 2020Geom-GCN: Geometric Graph Convolutional Networkscited by 821
  3. 2015Gated Graph Sequence Neural Networkscited by 792
  4. 2019DropEdge: Towards Deep Graph Convolutional Networks on Node Classificationcited by 679
  5. 2020On the Bottleneck of Graph Neural Networks and its Practical Implicationscited by 624
  6. 2020Adaptive Universal Generalized PageRank Graph Neural Networkcited by 592
  7. 2019Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological Viewcited by 544
  8. 2019Multi-scale Attributed Node Embeddingcited by 543
  9. 2021Understanding over-squashing and bottlenecks on graphs via curvaturecited by 471
  10. 2016Geometric deep learning on graphs and manifolds using mixture model CNNscited by 468
  11. 2019MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixingcited by 442
  12. 2021Beyond Low-frequency Information in Graph Convolutional Networkscited by 361
  13. 2017Residual Gated Graph ConvNetscited by 341
  14. 2021Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methodscited by 309
  15. 2019PairNorm: Tackling Oversmoothing in GNNscited by 297
  16. 2019Revisiting Graph Neural Networks: All We Have is Low-Pass Filterscited by 260
  17. 2021GRAND: Graph Neural Diffusioncited by 258
  18. 2021Two Sides of the Same Coin: Heterophily and Oversmoothing in Graph Convolutional Neural Networkscited by 182
  19. 2022Finding Global Homophily in Graph Neural Networks When Meeting Heterophilycited by 179
  20. 2018GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphscited by 164
  21. 2019Graph Neural Ordinary Differential Equationscited by 153
  22. 2020Multiscale Deep Equilibrium Modelscited by 149
  23. 2022Graph-Coupled Oscillator Networkscited by 123
  24. 2019Continuous Graph Neural Networkscited by 114
  25. 2021PDE-GCN: Novel Architectures for Graph Neural Networks Motivated by Partial Differential Equationscited by 106
  26. 2021Beltrami Flow and Neural Diffusion on Graphscited by 84
  27. 2021Is Heterophily A Real Nightmare For Graph Neural Networks To Do Node Classification?cited by 82
  28. 2021UnICORNN: A recurrent model for learning very long time dependenciescited by 56
  29. 2021Long Expressive Memory for Sequence Modelingcited by 41
  30. 2021Designing the Topology of Graph Neural Networks: A Novel Feature Fusion Perspectivecited by 19
  31. 2019Discrete and Continuous Deep Residual Learning Over Graphscited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 14; 2024: 16; 2025: 16; 2026: 2.

49 papers.

Sort:

By area:

  1. 2023A Survey on Oversmoothing in Graph Neural Networkscited by 246
  2. 2023Edge Directionality Improves Learning on Heterophilic Graphscited by 88
  3. 2023DRew: Dynamically Rewired Message Passing with Delaycited by 81
  4. 2024Polynormer: Polynomial-Expressive Graph Transformer in Linear Timecited by 64
  5. 2023How does over-squashing affect the power of GNNs?cited by 53
  6. 2025On Vanishing Gradients, Over-Smoothing, and Over-Squashing in GNNs: Bridging Recurrent and Graph Learningcited by 42
  7. 2023Cooperative Graph Neural Networkscited by 35
  8. 2023Graph Positional Encoding via Random Feature Propagationcited by 30
  9. 2023From Continuous Dynamics to Graph Neural Networks: Neural Diffusion and Beyondcited by 26
  10. 2023Provably Powerful Graph Neural Networks for Directed Multigraphscited by 24
  11. 2024Spectral Graph Pruning Against Over-Squashing and Over-Smoothingcited by 23
  12. 2023HoloNets: Spectral Convolutions do extend to Directed Graphscited by 20
37 more
  1. 2022Improving Graph Neural Networks with Learnable Propagation Operatorscited by 19
  2. 2023Rank Collapse Causes Over-Smoothing and Over-Correlation in Graph Neural Networkscited by 18
  3. 2023Feature Transportation Improves Graph Neural Networkscited by 15
  4. 2023Multi-Scale Message Passing Neural PDE Solverscited by 12
  5. 2024Understanding Oversmoothing in Diffusion-Based GNNs From the Perspective of Operator Semigroup Theorycited by 8
  6. 2023Supercharging Graph Transformers with Advective Diffusioncited by 7
  7. 2024Graph Neural Reaction Diffusion Modelscited by 6
  8. 2024Preventing Representational Rank Collapse in MPNNs by Splitting the Computational Graphcited by 6
  9. 2025Are We Measuring Oversmoothing in Graph Neural Networks Correctly?cited by 6
  10. 2025Deeper with Riemannian Geometry: Overcoming Oversmoothing and Oversquashing for Graph Foundation Modelscited by 6
  11. 2023G-Signatures: Global Graph Propagation With Randomized Signaturescited by 4
  12. 2024Oversmoothing Alleviation in Graph Neural Networks: A Survey and Unified Viewcited by 4
  13. 2024Simplifying the Theory on Over-Smoothingcited by 4
  14. 2024Improving Graph Neural Networks by Learning Continuous Edge Directionscited by 3
  15. 2025The Oversmoothing Fallacy: A Misguided Narrative in GNN Researchcited by 3
  16. 2025HeroFilter: Adaptive Spectral Graph Filter for Varying Heterophilic Relationscited by 3
  17. 2024Graph Neural Aggregation-diffusion with Metastabilitycited by 2
  18. 2024Taming Gradient Oversmoothing and Expansion in Graph Neural Networkscited by 2
  19. 2024Graph Neural Networks Need Cluster-Normalize-Activate Modulescited by 2
  20. 2025Efficient Learning on Large Graphs using a Densifying Regularity Lemmacited by 2
  21. 2025Topology-aware Neural Flux Prediction Guided by Physicscited by 2
  22. 2024Mixture of Scope Experts at Test: Generalizing Deeper Graph Neural Networks with Shallow Variantscited by 1
  23. 2024Transformers from Diffusion: A Unified Framework for Neural Message Passingcited by 1
  24. 2024A Dynamical Systems-Inspired Pruning Strategy for Addressing Oversmoothing in Graph Neural Networkscited by 1
  25. 2025What Can We Learn From MIMO Graph Convolutions?cited by 1
  26. 2025Measuring Over-smoothing beyond Dirichlet energycited by 1
  27. 2026Position: Don't be Afraid of Over-Smoothing And Over-Squashingcited by 1
  28. 2024Entropy Aware Message Passing in Graph Neural Networkscited by 0
  29. 2024Deep Graph Attention Networkscited by 0
  30. 2025Personalized Layer Selection for Graph Neural Networkscited by 0
  31. 2025Channel-Attentive Graph Neural Networkscited by 0
  32. 2025Using Subgraph GNNs for Node Classification:an Overlooked Potential Approachcited by 0
  33. 2025H3GNNs: Harmonizing Heterophily and Homophily in GNNs via Joint Structural Node Encoding and Self-Supervised Learningcited by 0
  34. 2025Directed Homophily-Aware Graph Neural Networkcited by 0
  35. 2025Topology-Assisted Spatio-Temporal Pattern Disentangling for Scalable MARL in Large-scale Autonomous Traffic Controlcited by 0
  36. 2025Solving Oversmoothing in GNNs via Nonlocal Message Passing: Algebraic Smoothing and Depth Scalabilitycited by 0
  37. 2026Revisiting Pre-Propagation GNNs: Robust Diffusion Operators and Hidden-State Re-Propagationcited by 0