Learning to Control the Smoothness of Graph Convolutional Network Features
Not on the map. Why?
Abstract (arXiv)
The pioneering work of Oono and Suzuki [ICLR, 2020] and Cai and Wang [arXiv:2006.13318] initializes the analysis of the smoothness of graph convolutional network (GCN) features. Their results reveal an intricate empirical correlation between node classification accuracy and the ratio of smooth to non-smooth feature components. However, the optimal ratio that favors node classification is unknown, and the non-smooth features of deep GCN with ReLU or leaky ReLU activation function diminish. In this paper, we propose a new strategy to let GCN learn node features with a desired smoothness -- adapting to data and tasks -- to enhance node classification. Our approach has three key steps: (1) We establish a geometric relationship between the input and output of ReLU or leaky ReLU. (2) Building on our geometric insights, we augment the message-passing process of graph convolutional layers (GCLs) with a learnable term to modulate the smoothness of node features with computational efficiency. (3) We investigate the achievable ratio between smooth and non-smooth feature components for GCNs with the augmented message-passing scheme. Our extensive numerical results show that the augmented message-passing schemes significantly improve node classification for GCN and some related models.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 30 of its 46 references that are in the map. First posted on arXiv 18 Oct 2024.
Builds on
30 of 46 references.
References · 30
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
- 2017Graph Attention Networkscited by 8,854
- 2017Inductive Representation Learning on Large Graphscited by 6,284
- 2017Neural Message Passing for Quantum Chemistrycited by 3,460
- 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
- 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
- 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,085
- 2016Identity Mappings in Deep Residual Networkscited by 1,960
- 2018Graph Neural Networks: A Review of Methods and Applicationscited by 1,571
- 2019Simplifying Graph Convolutional Networkscited by 1,470
- 2018Relational inductive biases, deep learning, and graph networkscited by 1,341
18 more
- 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,133
- 2018Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learningcited by 1,099
- 2016Revisiting Semi-Supervised Learning with Graph Embeddingscited by 970
- 2018Representation Learning on Graphs with Jumping Knowledge Networkscited by 875
- 2020Simple and Deep Graph Convolutional Networkscited by 857
- 2020Geom-GCN: Geometric Graph Convolutional Networkscited by 821
- 2019DropEdge: Towards Deep Graph Convolutional Networks on Node Classificationcited by 679
- 2019Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological Viewcited by 544
- 2017Representation Learning on Graphs: Methods and Applicationscited by 456
- 2019PairNorm: Tackling Oversmoothing in GNNscited by 297
- 2019Revisiting Graph Neural Networks: All We Have is Low-Pass Filterscited by 260
- 2021GRAND: Graph Neural Diffusioncited by 258
- 2020A Note on Over-Smoothing for Graph Neural Networkscited by 245
- 2020Implicit Graph Neural Networkscited by 98
- 2021Dirichlet Energy Constrained Learning for Deep Graph Neural Networkscited by 97
- 2020Scattering GCN: Overcoming Oversmoothness in Graph Convolutional Networkscited by 57
- 2022A Non-Asymptotic Analysis of Oversmoothing in Graph Neural Networkscited by 49
- 2018Mean-field theory of graph neural networks in graph partitioningcited by 16
Built on by
Not cited yet.