mlsanityData to 7 Oct 2026

Alleviating Over-Smoothing via Aggregation over Compact Manifolds

arXiv 2407.19231 ↗First posted 27 Jul 2024cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Graph neural networks (GNNs) have achieved significant success in various applications. Most GNNs learn the node features with information aggregation of its neighbors and feature transformation in each layer. However, the node features become indistinguishable after many layers, leading to performance deterioration: a significant limitation known as over-smoothing. Past work adopted various techniques for addressing this issue, such as normalization and skip-connection of layer-wise output. After the study, we found that the information aggregations in existing work are all contracted aggregations, with the intrinsic property that features will inevitably converge to the same single point after many layers. To this end, we propose the aggregation over compacted manifolds method (ACM) that replaces the existing information aggregation with aggregation over compact manifolds, a special type of manifold, which avoids contracted aggregations. In this work, we theoretically analyze contracted aggregation and its properties. We also provide an extensive empirical evaluation that shows ACM can effectively alleviate over-smoothing and outperforms the state-of-the-art. The code can be found in https://github.com/DongzhuoranZhou/ACM.git.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 39 of its 52 references that are in the map. First posted on arXiv 27 Jul 2024.

Builds on

39 of 52 references.

References · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  4. 2017Graph Attention Networkscited by 8,854
  5. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  6. 2017Inductive Representation Learning on Large Graphscited by 6,284
  7. 2018How Powerful are Graph Neural Networks?cited by 4,355
  8. 2018Graph Neural Networks: A Review of Methods and Applicationscited by 1,571
  9. 2019Simplifying Graph Convolutional Networkscited by 1,470
  10. 2018Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learningcited by 1,099
  11. 2018Pitfalls of Graph Neural Network Evaluationcited by 994
  12. 2018Predict then Propagate: Graph Neural Networks meet Personalized PageRankcited by 987
27 more
  1. 2016Revisiting Semi-Supervised Learning with Graph Embeddingscited by 970
  2. 2018Representation Learning on Graphs with Jumping Knowledge Networkscited by 875
  3. 2020Simple and Deep Graph Convolutional Networkscited by 857
  4. 2020Geom-GCN: Geometric Graph Convolutional Networkscited by 821
  5. 2019DropEdge: Towards Deep Graph Convolutional Networks on Node Classificationcited by 679
  6. 2020Adaptive Universal Generalized PageRank Graph Neural Networkcited by 592
  7. 2019Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological Viewcited by 544
  8. 2019DeepGCNs: Can GCNs Go as Deep as CNNs?cited by 527
  9. 2019Hyperbolic Graph Convolutional Neural Networkscited by 400
  10. 2018Hyperbolic Neural Networkscited by 393
  11. 2019PairNorm: Tackling Oversmoothing in GNNscited by 297
  12. 2020Towards Deeper Graph Neural Networkscited by 296
  13. 2020A Note on Over-Smoothing for Graph Neural Networkscited by 245
  14. 2019Hyperbolic Image Embeddingscited by 219
  15. 2019Multi-relational Poincar'e Graph Embeddingscited by 126
  16. 2018Hyperbolic Attention Networkscited by 122
  17. 2018Large-Scale Learnable Graph Convolutional Networkscited by 113
  18. 2020Towards Deeper Graph Neural Networks with Differentiable Group Normalizationcited by 113
  19. 2019Constant Curvature Graph Convolutional Networkscited by 106
  20. 2020Tackling Over-Smoothing for General Graph Convolutional Networkscited by 66
  21. 2022Self-Supervised Continual Graph Learning in Adaptive Riemannian Spacescited by 32
  22. 2024Motif-aware Riemannian Graph Neural Network with Generative-Contrastive Learningcited by 23
  23. 2022Feature Overcorrelation in Deep Graph Neural Networks: A New Perspectivecited by 19
  24. 2022Stability of Aggregation Graph Neural Networkscited by 7
  25. 2023Neural Snowflakes: Universal Latent Graph Inference via Trainable Latent Geometriescited by 6
  26. 2022Generalizing Aggregation Functions in GNNs:High-Capacity GNNs via Nonlinear Neighborhood Aggregatorscited by 3
  27. 2023SCV-GNN: Sparse Compressed Vector-based Graph Neural Network Aggregationcited by 2

Built on by

Not cited yet.