mlsanityData to 7 Oct 2026

Dirichlet Energy Constrained Learning for Deep Graph Neural Networks

arXiv 2107.02392 ↗First posted 6 Jul 2021cs.LGcited by 97

Not on the map. Why?

Abstract (arXiv)

Graph neural networks (GNNs) integrate deep architectures and topological structure modeling in an effective way. However, the performance of existing GNNs would decrease significantly when they stack many layers, because of the over-smoothing issue. Node embeddings tend to converge to similar vectors when GNNs keep recursively aggregating the representations of neighbors. To enable deep GNNs, several methods have been explored recently. But they are developed from either techniques in convolutional neural networks or heuristic strategies. There is no generalizable and theoretical principle to guide the design of deep GNNs. To this end, we analyze the bottleneck of deep GNNs by leveraging the Dirichlet energy of node embeddings, and propose a generalizable principle to guide the training of deep GNNs. Based on it, a novel deep GNN framework -- EGNN is designed. It could provide lower and upper constraints in terms of Dirichlet energy at each layer to avoid over-smoothing. Experimental results demonstrate that EGNN achieves state-of-the-art performance by using deep layers.

Key facts. Cited by 97 papers in the map (data through 7 Oct 2026). Builds on 49 of its 55 references that are in the map. First posted on arXiv 6 Jul 2021.

Builds on

49 of 55 references.

References · 49

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  3. 2017Graph Attention Networkscited by 8,854
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  5. 2017Inductive Representation Learning on Large Graphscited by 6,284
  6. 2018How Powerful are Graph Neural Networks?cited by 4,355
  7. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  8. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  9. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
  10. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  11. 2019Simplifying Graph Convolutional Networkscited by 1,470
  12. 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,133
37 more
  1. 2018Pitfalls of Graph Neural Network Evaluationcited by 994
  2. 2018Predict then Propagate: Graph Neural Networks meet Personalized PageRankcited by 987
  3. 2016Revisiting Semi-Supervised Learning with Graph Embeddingscited by 970
  4. 2018Representation Learning on Graphs with Jumping Knowledge Networkscited by 875
  5. 2020Simple and Deep Graph Convolutional Networkscited by 857
  6. 2018Link Prediction Based on Graph Neural Networkscited by 829
  7. 2015Gated Graph Sequence Neural Networkscited by 792
  8. 2019Graph Neural Networks for Social Recommendationcited by 711
  9. 2019DropEdge: Towards Deep Graph Convolutional Networks on Node Classificationcited by 679
  10. 2020On the Bottleneck of Graph Neural Networks and its Practical Implicationscited by 624
  11. 2020Adaptive Universal Generalized PageRank Graph Neural Networkcited by 592
  12. 2019Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological Viewcited by 544
  13. 2019DeepGCNs: Can GCNs Go as Deep as CNNs?cited by 527
  14. 2019Graph U-Netscited by 489
  15. 2019PairNorm: Tackling Oversmoothing in GNNscited by 297
  16. 2020Towards Deeper Graph Neural Networkscited by 296
  17. 2019Revisiting Graph Neural Networks: All We Have is Low-Pass Filterscited by 260
  18. 2020DeeperGCN: All You Need to Train Deeper GCNscited by 249
  19. 2020A Note on Over-Smoothing for Graph Neural Networkscited by 245
  20. 2020Graph Random Neural Network for Semi-Supervised Learning on Graphscited by 213
  21. 2020Iterative Deep Graph Learning for Graph Neural Networks: Better and Robust Node Embeddingscited by 212
  22. 2017Predicting multicellular function through multi-layer tissue networkscited by 138
  23. 2017Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practicecited by 132
  24. 2015A Simple Way to Initialize Recurrent Networks of Rectified Linear Unitscited by 130
  25. 2020Revisiting Graph based Collaborative Filtering: A Linear Residual Graph Convolutional Network Approachcited by 119
  26. 2018Large-Scale Learnable Graph Convolutional Networkscited by 113
  27. 2020Towards Deeper Graph Neural Networks with Differentiable Group Normalizationcited by 113
  28. 2021Orthogonalizing Convolutional Layers with the Cayley Transformcited by 105
  29. 2017Geometric Matrix Completion with Recurrent Multi-Graph Neural Networkscited by 88
  30. 2020Bayesian Graph Neural Networks with Adaptive Connection Samplingcited by 81
  31. 2019Auto-GNN: Neural Architecture Search of Graph Neural Networkscited by 73
  32. 2020Tackling Over-Smoothing for General Graph Convolutional Networkscited by 66
  33. 2019GraphZoom: A multi-level spectral approach for accurate and scalable graph embeddingcited by 58
  34. 2020Understanding and Resolving Performance Degradation in Graph Convolutional Networkscited by 53
  35. 2020Deep Isometric Learning for Visual Recognitioncited by 38
  36. 2020Controllable Orthogonalization in Training DNNscited by 30
  37. 2017On the Effects of Batch and Weight Normalization in Generative Adversarial Networkscited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 16; 2023: 13; 2024: 34; 2025: 14; 2026: 13.

97 papers.

Sort:

By area:

  1. 2023A Survey on Oversmoothing in Graph Neural Networkscited by 246
  2. 2023Data-centric Artificial Intelligence: A Surveycited by 202
  3. 2022Convolutional Neural Networks on Graphs with Chebyshev Approximation, Revisitedcited by 87
  4. 2022Understanding convolution on graphs via energiescited by 58
  5. 2022Anti-Symmetric DGN: a stable architecture for Deep Graph Networkscited by 58
  6. 2022A Comprehensive Study on Large-Scale Graph Training: Benchmarking and Rethinkingcited by 56
  7. 2021Bag of Tricks for Training Deeper Graph Neural Networks: A Comprehensive Benchmark Studycited by 36
  8. 2024Kolmogorov-Arnold Graph Neural Networkscited by 31
  9. 2023PC-Conv: Unifying Homophily and Heterophily with Two-fold Filteringcited by 28
  10. 2022Transformers from an Optimization Perspectivecited by 27
  11. 2024How Universal Polynomial Bases Enhance Spectral Graph Neural Networks: Heterophily, Over-smoothing, and Over-squashingcited by 24
  12. 2024Spectral Graph Pruning Against Over-Squashing and Over-Smoothingcited by 23
38 more
  1. 2024Optimizing OOD Detection in Molecular Graphs: A Novel Approach with Diffusion Modelscited by 21
  2. 2021Orthogonal Graph Neural Networkscited by 19
  3. 2022Improving Graph Neural Networks with Learnable Propagation Operatorscited by 19
  4. 2023Rank Collapse Causes Over-Smoothing and Over-Correlation in Graph Neural Networkscited by 18
  5. 2025Oversmoothing, Oversquashing, Heterophily, Long-Range, and more: Demystifying Common Beliefs in Graph Machine Learningcited by 18
  6. 2021CAP: Co-Adversarial Perturbation on Weights and Features for Improving Generalization of Graph Neural Networkscited by 15
  7. 2022ResNorm: Tackling Long-tailed Degree Distribution Issue in Graph Neural Networks via Normalizationcited by 15
  8. 2022pathGCN: Learning General Graph Spatial Operators from Pathscited by 15
  9. 2025Uncertainty-Aware Graph Neural Networks: A Multi-Hop Evidence Fusion Approachcited by 14
  10. 2022Towards Similarity-Aware Time-Series Classificationcited by 13
  11. 2024Investigating Out-of-Distribution Generalization of GNNs: An Architecture Perspectivecited by 13
  12. 2025GNNs Getting ComFy: Community and Feature Similarity Guided Rewiringcited by 13
  13. 2021Adaptive Label Smoothing To Regularize Large-Scale Graph Trainingcited by 12
  14. 2022Old can be Gold: Better Gradient Flow can Make Vanilla-GCNs Great Againcited by 11
  15. 2024Masked Graph Autoencoder with Non-discrete Bandwidthscited by 11
  16. 2021SkipNode: On Alleviating Performance Degradation for Deep Graph Convolutional Networkscited by 10
  17. 2024Beyond Over-smoothing: Uncovering the Trainability Challenges in Deep Graph Neural Networkscited by 10
  18. 2023D4Explainer: In-Distribution GNN Explanations via Discrete Denoising Diffusioncited by 9
  19. 2024GTC: GNN-Transformer Co-contrastive Learning for Self-supervised Heterogeneous Graph Representationcited by 9
  20. 2022Convergence of Invariant Graph Networkscited by 8
  21. 2024Sign is Not a Remedy: Multiset-to-Multiset Message Passing for Learning on Heterophilic Graphscited by 8
  22. 2021A Piece-wise Polynomial Filtering Approach for Graph Neural Networkscited by 7
  23. 2022Towards Training GNNs using Explanation Directed Message Passingcited by 7
  24. 2024Efficient Sharpness-Aware Minimization for Molecular Graph Transformer Modelscited by 7
  25. 2024Preventing Representational Rank Collapse in MPNNs by Splitting the Computational Graphcited by 6
  26. 2025Are We Measuring Oversmoothing in Graph Neural Networks Correctly?cited by 6
  27. 2025What makes a good feedforward computational graph?cited by 5
  28. 2022Alternately Optimized Graph Neural Networkscited by 4
  29. 2023DeepHGCN: Toward Deeper Hyperbolic Graph Convolutional Networkscited by 4
  30. 2024Oversmoothing Alleviation in Graph Neural Networks: A Survey and Unified Viewcited by 4
  31. 2024Rethinking Independent Cross-Entropy Loss For Graph-Structured Datacited by 4
  32. 2024Global-Local Graph Neural Networks for Node-Classificationcited by 4
  33. 2024Simplifying the Theory on Over-Smoothingcited by 4
  34. 2025Network-wide Freeway Traffic Estimation Using Sparse Sensor Data: A Dirichlet Graph Auto-Encoder Approachcited by 4
  35. 2023Steering Graph Neural Networks with Pinning Controlcited by 3
  36. 2023Towards Label Position Bias in Graph Neural Networkscited by 3
  37. 2023Dirichlet Energy Enhancement of Graph Neural Networks by Framelet Augmentationcited by 3
  38. 2024Simplified PCNet with Robustnesscited by 3

50 most cited of 97 All 97 (JSON)