mlsanityData to 7 Oct 2026

Mitigating Oversmoothing Through Reverse Process of GNNs for Heterophilic Graphs

arXiv 2403.10543 ↗First posted 11 Mar 2024cs.SI cs.LGcited by 10

Not on the map. Why?

Abstract (arXiv)

Graph Neural Network (GNN) resembles the diffusion process, leading to the over-smoothing of learned representations when stacking many layers. Hence, the reverse process of message passing can produce the distinguishable node representations by inverting the forward message propagation. The distinguishable representations can help us to better classify neighboring nodes with different labels, such as in heterophilic graphs. In this work, we apply the design principle of the reverse process to the three variants of the GNNs. Through the experiments on heterophilic graph data, where adjacent nodes need to have different representations for successful classification, we show that the reverse process significantly improves the prediction performance in many cases. Additional analysis reveals that the reverse mechanism can mitigate the over-smoothing over hundreds of layers. Our code is available at https://github.com/ml-postech/reverse-gnn.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 28 of its 34 references that are in the map. First posted on arXiv 11 Mar 2024.

Builds on

28 of 34 references.

References · 28

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  3. 2017Graph Attention Networkscited by 8,854
  4. 2017Inductive Representation Learning on Large Graphscited by 6,284
  5. 2018Neural Ordinary Differential Equationscited by 3,733
  6. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,085
  7. 2018Predict then Propagate: Graph Neural Networks meet Personalized PageRankcited by 987
  8. 2019DropEdge: Towards Deep Graph Convolutional Networks on Node Classificationcited by 679
  9. 2020Adaptive Universal Generalized PageRank Graph Neural Networkcited by 592
  10. 2019Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological Viewcited by 544
  11. 2020Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classificationcited by 500
  12. 2019MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixingcited by 442
16 more
  1. 2018Invertible Residual Networkscited by 364
  2. 2021Beyond Low-frequency Information in Graph Convolutional Networkscited by 361
  3. 2023A critical look at the evaluation of GNNs under heterophily: Are we really making progress?cited by 332
  4. 2019PairNorm: Tackling Oversmoothing in GNNscited by 297
  5. 2023A Survey on Oversmoothing in Graph Neural Networkscited by 246
  6. 2021BernNet: Learning Arbitrary Graph Spectral Filters via Bernstein Approximationcited by 195
  7. 2022Finding Global Homophily in Graph Neural Networks When Meeting Heterophilycited by 179
  8. 2022How Powerful are Spectral Graph Neural Networkscited by 169
  9. 2022Revisiting Heterophily For Graph Neural Networkscited by 169
  10. 2023Edge Directionality Improves Learning on Heterophilic Graphscited by 88
  11. 2021GBK-GNN: Gated Bi-Kernel Graph Neural Networks for Modeling Both Homophily and Heterophilycited by 84
  12. 2023Ordered GNN: Ordering Message Passing to Deal with Heterophily and Over-smoothingcited by 67
  13. 2022Characterizing Graph Datasets for Node Classification: Homophily-Heterophily Dichotomy and Beyondcited by 60
  14. 2021Simplifying approach to Node Classification in Graph Neural Networkscited by 56
  15. 2022Model Degradation Hinders Deep Graph Neural Networkscited by 26
  16. 2023Half-Hop: A graph upsampling approach for slowing down message passingcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 2; 2026: 6.

10 papers.

  1. 2025The Oversmoothing Fallacy: A Misguided Narrative in GNN Researchcited by 3
  2. 2024Taming Gradient Oversmoothing and Expansion in Graph Neural Networkscited by 2
  3. 2024Making Classic GNNs Strong Baselines Across Varying Homophily: A Smoothness-Generalization Perspectivecited by 2
  4. 2025Towards a deeper GCN: Alleviate over-smoothing with iterative training and fine-tuningcited by 0
  5. 2026Reversible Residual Normalization Alleviates Spatio-Temporal Distribution Shiftcited by 0
  6. 2026GOMA: Toward Structure-Driven Multimodal Alignment from a Graph Signal Smoothing Perspectivecited by 0
  7. 2026Graph Transductive Sharpening: Leveraging Unlabeled Predictions in Node Classificationcited by 0
  8. 2026Learn When and Where to Connect: Adaptive Virtual Nodes for Dynamic Message Passing on Graphscited by 0
  9. 2026A Spectral Theory of Normalized Corrected GNN Propagationcited by 0
  10. 2026ProbeMatchDTI: Probe-Driven Multi-Scale Biochemical Pattern Matching for Drug-Target Interaction Predictioncited by 0