A Dynamical Systems-Inspired Pruning Strategy for Addressing Oversmoothing in Graph Neural Networks
Not on the map. Why?
Abstract (arXiv)
Oversmoothing in Graph Neural Networks (GNNs) poses a significant challenge as network depth increases, leading to homogenized node representations and a loss of expressiveness. In this work, we approach the oversmoothing problem from a dynamical systems perspective, providing a deeper understanding of the stability and convergence behavior of GNNs. Leveraging insights from dynamical systems theory, we identify the root causes of oversmoothing and propose textbftextitDYNAMO-GAT. This approach utilizes noise-driven covariance analysis and Anti-Hebbian principles to selectively prune redundant attention weights, dynamically adjusting the network's behavior to maintain node feature diversity and stability. Our theoretical analysis reveals how DYNAMO-GAT disrupts the convergence to oversmoothed states, while experimental results on benchmark datasets demonstrate its superior performance and efficiency compared to traditional and state-of-the-art methods. DYNAMO-GAT not only advances the theoretical understanding of oversmoothing through the lens of dynamical systems but also provides a practical and effective solution for improving the stability and expressiveness of deep GNNs.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 35 of its 43 references that are in the map. First posted on arXiv 10 Dec 2024.
Builds on
35 of 43 references.
References · 35
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
- 2017Graph Attention Networkscited by 8,923
- 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
- 2016Layer Normalizationcited by 5,669
- 2018How Powerful are Graph Neural Networks?cited by 4,393
- 2017Neural Message Passing for Quantum Chemistrycited by 3,487
- 2019A Comprehensive Survey on Graph Neural Networkscited by 2,919
- 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,138
- 2018Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learningcited by 1,102
- 2018Representation Learning on Graphs with Jumping Knowledge Networkscited by 877
- 2019Graph Neural Networks for Social Recommendationcited by 714
- 2019Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological Viewcited by 545
23 more
- 2019DeepGCNs: Can GCNs Go as Deep as CNNs?cited by 527
- 2021GemNet: Universal Directional Graph Neural Networks for Moleculescited by 343
- 2020A Note on Over-Smoothing for Graph Neural Networkscited by 249
- 2023A Survey on Oversmoothing in Graph Neural Networkscited by 247
- 2022Graph neural networks for materials science and chemistrycited by 182
- 2022Graph-Coupled Oscillator Networkscited by 124
- 2020Towards Deeper Graph Neural Networks with Differentiable Group Normalizationcited by 114
- 2022Not too little, not too much: a theoretical analysis of graph (over)smoothingcited by 106
- 2023How does over-squashing affect the power of GNNs?cited by 53
- 2022Gradient Gating for Deep Multi-Rate Learning on Graphscited by 49
- 2023Demystifying Oversmoothing in Attention-Based Graph Neural Networkscited by 49
- 2021A Fully Spiking Hybrid Neural Network for Energy-Efficient Object Detectioncited by 31
- 2022Heterogeneous Recurrent Spiking Neural Network for Spatio-Temporal Classificationcited by 11
- 2024Sparse Spiking Neural Network: Exploiting Heterogeneity in Timescales for Pruning Recurrent SNNcited by 10
- 2021Characterization of Generalizability of Spike Timing Dependent Plasticity trained Spiking Neural Networkscited by 8
- 2023Heterogeneous Neuronal and Synaptic Dynamics for Spike-Efficient Unsupervised Learning: Theory and Design Principlescited by 8
- 2020Using noise to probe recurrent neural network structure and prune synapsescited by 5
- 2023Brain-Inspired Spiking Neural Network for Online Unsupervised Time Series Predictioncited by 5
- 2024Simplifying the Theory on Over-Smoothingcited by 4
- 2024Topological Representations of Heterogeneous Learning Dynamics of Recurrent Spiking Neural Networkscited by 3
- 2021μDARTS: Model Uncertainty-Aware Differentiable Architecture Searchcited by 2
- 2023Unsupervised 3D Object Learning through Neuron Activity aware Plasticitycited by 2
- 2024Exploiting Heterogeneity in Timescales for Sparse Recurrent Spiking Neural Networks for Energy-Efficient Edge Computingcited by 2
Built on by
Citing papers per year: 2025: 1.
1 paper.