mlsanityData to 14 Oct 2026

Feature Overcorrelation in Deep Graph Neural Networks: A New Perspective

arXiv 2206.07743 ↗First posted 15 Jun 2022cs.LG cs.AIcited by 19

Not on the map. Why?

Abstract (arXiv)

Recent years have witnessed remarkable success achieved by graph neural networks (GNNs) in many real-world applications such as recommendation and drug discovery. Despite the success, oversmoothing has been identified as one of the key issues which limit the performance of deep GNNs. It indicates that the learned node representations are highly indistinguishable due to the stacked aggregators. In this paper, we propose a new perspective to look at the performance degradation of deep GNNs, i.e., feature overcorrelation. Through empirical and theoretical study on this matter, we demonstrate the existence of feature overcorrelation in deeper GNNs and reveal potential reasons leading to this issue. To reduce the feature correlation, we propose a general framework DeCorr which can encourage GNNs to encode less redundant information. Extensive experiments have demonstrated that DeCorr can help enable deeper GNNs and is complementary to existing techniques tackling the oversmoothing issue.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 40 of its 53 references that are in the map. First posted on arXiv 15 Jun 2022.

Builds on

40 of 53 references.

References · 40

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  3. 2017Graph Attention Networkscited by 8,923
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  5. 2017Inductive Representation Learning on Large Graphscited by 6,320
  6. 2017Neural Message Passing for Quantum Chemistrycited by 3,487
  7. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,919
  8. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,434
  9. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,089
  10. 2018Graph Neural Networks: A Review of Methods and Applicationscited by 1,576
  11. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,209
  12. 2013Spectral Networks and Locally Connected Networks on Graphscited by 1,102
28 more
  1. 2018Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learningcited by 1,102
  2. 2018Pitfalls of Graph Neural Network Evaluationcited by 1,007
  3. 2020Simple and Deep Graph Convolutional Networkscited by 860
  4. 2020Geom-GCN: Geometric Graph Convolutional Networkscited by 826
  5. 2015Convolutional Networks on Graphs for Learning Molecular Fingerprintscited by 754
  6. 2019Graph Neural Networks for Social Recommendationcited by 714
  7. 2019DropEdge: Towards Deep Graph Convolutional Networks on Node Classificationcited by 680
  8. 2019Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological Viewcited by 545
  9. 2019DeepGCNs: Can GCNs Go as Deep as CNNs?cited by 527
  10. 2019PairNorm: Tackling Oversmoothing in GNNscited by 297
  11. 2020Towards Deeper Graph Neural Networkscited by 296
  12. 2021Graph Contrastive Learning Automatedcited by 287
  13. 2015Deep Convolutional Networks on Graph-Structured Datacited by 246
  14. 2018Modeling polypharmacy side effects with graph convolutional networkscited by 237
  15. 2020Data Augmentation for Graph Neural Networkscited by 203
  16. 2021Training Graph Neural Networks with 1000 Layerscited by 169
  17. 2022Data Augmentation for Deep Graph Learning: A Surveycited by 133
  18. 2020Towards Deeper Graph Neural Networks with Differentiable Group Normalizationcited by 114
  19. 2015Reducing Overfitting in Deep Networks by Decorrelating Representationscited by 94
  20. 2021Few-Shot Graph Learning for Molecular Property Predictioncited by 81
  21. 2021Elastic Graph Neural Networkscited by 67
  22. 2022Graph Data Augmentation for Graph Machine Learning: A Surveycited by 62
  23. 2022Improving Fairness in Graph Neural Networks via Mitigating Sensitive Attribute Leakagecited by 61
  24. 2021Automated Self-Supervised Learning for Graphscited by 50
  25. 2016Regularizing CNNs with Locally Constrained Decorrelationscited by 41
  26. 2018Exploiting Semantics in Neural Machine Translation with Graph Convolutional Networkscited by 30
  27. 2022Graph Neural Networks for Multimodal Single-Cell Data Integrationcited by 18
  28. 2019On Correlation of Features Extracted by Deep Neural Networkscited by 4

Built on by

Cited by year

Citing papers per year: 2023: 7; 2024: 7; 2025: 5.

19 papers.

Sort:

  1. 2023PC-Conv: Unifying Homophily and Heterophily with Two-fold Filteringcited by 28
  2. 2023Towards Deep Attention in Graph Neural Networks: Problems and Remediescited by 24
  3. 2023Rank Collapse Causes Over-Smoothing and Over-Correlation in Graph Neural Networkscited by 18
  4. 2024Graph Neural Networks for Tabular Data Learning: A Survey with Taxonomy and Directionscited by 13
  5. 2024Preventing Representational Rank Collapse in MPNNs by Splitting the Computational Graphcited by 6
  6. 2023Towards Better Graph Representation Learning with Parameterized Decomposition & Filteringcited by 4
  7. 2023Deep Graph Neural Networks via Posteriori-Sampling-based Node-Adaptive Residual Modulecited by 3
  8. 2024Infinite-Horizon Graph Filters: Leveraging Power Series to Enhance Sparse Information Aggregationcited by 2
  9. 2024Generalized Probabilistic Attention Mechanism in Transformerscited by 2
  10. 2024Investigating Graph Neural Networks and Classical Feature-Extraction Techniques in Activity-Cliff and Molecular Property Predictioncited by 2
  11. 2025What Can We Learn From MIMO Graph Convolutions?cited by 1
  12. 2025Understanding and Tackling Over-Dilution in Graph Neural Networkscited by 1
7 more
  1. 2025Enhancing Node-Level Graph Domain Adaptation by Alleviating Local Dependencycited by 1
  2. 2023Adaptive Depth Graph Attention Networkscited by 0
  3. 2023Asymmetric Learning for Graph Neural Network based Link Predictioncited by 0
  4. 2024GEN: A Practical Alternative to Graph Transformers for Long-Range Graph Modelingcited by 0
  5. 2024Alleviating Over-Smoothing via Aggregation over Compact Manifoldscited by 0
  6. 2025Improving the Stability of GNN Force Field Models by Reducing Feature Correlationcited by 0
  7. 2025Rethinking Message Passing Neural Networks with Diffusion Distance-guided Stress Majorizationcited by 0