mlsanityData to 7 Oct 2026

Graph Neural Aggregation-diffusion with Metastability

arXiv 2403.20221 ↗First posted 29 Mar 2024cs.LG cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Continuous graph neural models based on differential equations have expanded the architecture of graph neural networks (GNNs). Due to the connection between graph diffusion and message passing, diffusion-based models have been widely studied. However, diffusion naturally drives the system towards an equilibrium state, leading to issues like over-smoothing. To this end, we propose GRADE inspired by graph aggregation-diffusion equations, which includes the delicate balance between nonlinear diffusion and aggregation induced by interaction potentials. The node representations obtained through aggregation-diffusion equations exhibit metastability, indicating that features can aggregate into multiple clusters. In addition, the dynamics within these clusters can persist for long time periods, offering the potential to alleviate over-smoothing effects. This nonlinear diffusion in our model generalizes existing diffusion-based models and establishes a connection with classical GNNs. We prove that GRADE achieves competitive performance across various benchmarks and alleviates the over-smoothing issue in GNNs evidenced by the enhanced Dirichlet energy.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 49 of its 80 references that are in the map. First posted on arXiv 29 Mar 2024.

Builds on

49 of 80 references.

References · 49

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  2. 2017Graph Attention Networkscited by 8,854
  3. 2017Inductive Representation Learning on Large Graphscited by 6,284
  4. 2018Neural Ordinary Differential Equationscited by 3,733
  5. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  6. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  7. 2019Simplifying Graph Convolutional Networkscited by 1,470
  8. 2018Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learningcited by 1,099
  9. 2018Pitfalls of Graph Neural Network Evaluationcited by 994
  10. 2018Representation Learning on Graphs with Jumping Knowledge Networkscited by 875
  11. 2020Simple and Deep Graph Convolutional Networkscited by 857
  12. 2020Geom-GCN: Geometric Graph Convolutional Networkscited by 821
37 more
  1. 2015Image-based Recommendations on Styles and Substitutescited by 702
  2. 2020Adaptive Universal Generalized PageRank Graph Neural Networkcited by 592
  3. 2019Multi-scale Attributed Node Embeddingcited by 543
  4. 2016Geometric deep learning on graphs and manifolds using mixture model CNNscited by 468
  5. 2019MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixingcited by 442
  6. 2021Beyond Low-frequency Information in Graph Convolutional Networkscited by 361
  7. 2021Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methodscited by 309
  8. 2020Learning from Protein Structure with Geometric Vector Perceptronscited by 308
  9. 2019PairNorm: Tackling Oversmoothing in GNNscited by 297
  10. 2021Combinatorial optimization and reasoning with graph neural networkscited by 267
  11. 2020Scalable Gradients for Stochastic Differential Equationscited by 264
  12. 2021GRAND: Graph Neural Diffusioncited by 258
  13. 2023A Survey on Oversmoothing in Graph Neural Networkscited by 246
  14. 2022Neural Sheaf Diffusion: A Topological Perspective on Heterophily and Oversmoothing in GNNscited by 208
  15. 2017Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equationscited by 203
  16. 2018Deep Neural Networks Motivated by Partial Differential Equationscited by 187
  17. 2021Two Sides of the Same Coin: Heterophily and Oversmoothing in Graph Convolutional Neural Networkscited by 182
  18. 2022Finding Global Homophily in Graph Neural Networks When Meeting Heterophilycited by 179
  19. 2022Revisiting Heterophily For Graph Neural Networkscited by 169
  20. 2018Contextual Stochastic Block Modelscited by 134
  21. 2022Graph-Coupled Oscillator Networkscited by 123
  22. 2019Continuous Graph Neural Networkscited by 114
  23. 2021Geometric Deep Learning on Molecular Representationscited by 113
  24. 2021Combinatorial Optimization with Physics-Inspired Graph Neural Networkscited by 107
  25. 2021PDE-GCN: Novel Architectures for Graph Neural Networks Motivated by Partial Differential Equationscited by 106
  26. 2021Breaking the Limit of Graph Neural Networks by Improving the Assortativity of Graphs with Local Mixing Patternscited by 95
  27. 2023DIFFormer: Scalable (Graph) Transformers Induced by Energy Constrained Diffusioncited by 85
  28. 2021Beltrami Flow and Neural Diffusion on Graphscited by 84
  29. 2019Deep Learning via Dynamical Systems: An Approximation Perspectivecited by 78
  30. 2020SDE-Net: Equipping Deep Neural Networks with Uncertainty Estimatescited by 75
  31. 2019LanczosNet: Multi-Scale Deep Graph Convolutional Networkscited by 73
  32. 2020STEER: Simple Temporal Regularization For Neural ODEscited by 58
  33. 2021Dissecting the Diffusion Process in Linear Graph Convolutional Networkscited by 58
  34. 2022GREAD: Graph Neural Reaction-Diffusion Networkscited by 54
  35. 2022Gradient Gating for Deep Multi-Rate Learning on Graphscited by 49
  36. 2018Estimate and Replace: A Novel Approach to Integrating Deep Neural Networks with Existing Applicationscited by 21
  37. 2022TIDE: Time Derivative Diffusion for Deep Learning on Graphscited by 17

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 1.

2 papers.

  1. 2025Graph ODEs and Beyond: A Comprehensive Survey on Integrating Differential Equations with Graph Neural Networkscited by 14
  2. 2024S-Diff: An Anisotropic Diffusion Model for Collaborative Filtering in Spectral Domaincited by 5