mlsanityData to 14 Oct 2026

Neighbor-Sampling Based Momentum Stochastic Methods for Training Graph Neural Networks

arXiv 2508.00267 ↗First posted 1 Aug 2025math.OC cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Graph convolutional networks (GCNs) are a powerful tool for graph representation learning. Due to the recursive neighborhood aggregations employed by GCNs, efficient training methods suffer from a lack of theoretical guarantees or are missing important practical elements from modern deep learning algorithms, such as adaptivity and momentum. In this paper, we present several neighbor-sampling (NS) based Adam-type stochastic methods for solving a nonconvex GCN training problem. We utilize the control variate technique proposed by [1] to reduce the stochastic error caused by neighbor sampling. Under standard assumptions for Adam-type methods, we show that our methods enjoy the optimal convergence rate. In addition, we conduct extensive numerical experiments on node classification tasks with several benchmark datasets. The results demonstrate superior performance of our methods over classic NS-based SGD that also uses the control-variate technique, especially for large-scale graph datasets. Our code is available at https://github.com/RPI-OPT/CV-ADAM-GNN .

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 39 of its 59 references that are in the map. First posted on arXiv 1 Aug 2025.

Builds on

39 of 59 references.

References · 39

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  4. 2017Inductive Representation Learning on Large Graphscited by 6,320
  5. 2017Neural Message Passing for Quantum Chemistrycited by 3,487
  6. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,919
  7. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  8. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,089
  9. 2013Spectral Networks and Locally Connected Networks on Graphscited by 1,102
  10. 2019On the Convergence of Adam and Beyondcited by 841
  11. 2015Gated Graph Sequence Neural Networkscited by 796
  12. 2019Graph Neural Networks for Social Recommendationcited by 714
27 more
  1. 2019GraphSAINT: Graph Sampling Based Inductive Learning Methodcited by 618
  2. 2018FastGCN: Fast Learning with Graph Convolutional Networks via Importance Samplingcited by 584
  3. 2019Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networkscited by 541
  4. 2019Lower Bounds for Non-Convex Stochastic Optimizationcited by 307
  5. 2020DeeperGCN: All You Need to Train Deeper GCNscited by 250
  6. 2015Deep Convolutional Networks on Graph-Structured Datacited by 246
  7. 2017CayleyNets: Graph Convolutional Neural Networks with Complex Rational Spectral Filterscited by 203
  8. 2018Adaptive Sampling Towards Fast Graph Representation Learningcited by 180
  9. 2019Layer-Dependent Importance Sampling for Training Deep and Large Graph Convolutional Networkscited by 177
  10. 2018On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimizationcited by 174
  11. 2021Training Graph Neural Networks with 1000 Layerscited by 169
  12. 2018Adaptive Graph Convolutional Neural Networkscited by 155
  13. 2018Large-Scale Learnable Graph Convolutional Networkscited by 113
  14. 2020On the Convergence of SGD with Biased Gradientscited by 83
  15. 2019Graph Representation Learning: A Surveycited by 75
  16. 2022A Survey of Graph Neural Networks for Social Recommender Systemscited by 65
  17. 2020Bi-GCN: Binary Graph Convolutional Networkcited by 28
  18. 2014Block stochastic gradient iteration for convex and nonconvex optimizationcited by 22
  19. 2022Spatial-Temporal Interactive Dynamic Graph Convolution Network for Traffic Forecastingcited by 14
  20. 2021GIST: Distributed Training for Large-Scale Graph Convolutional Networkscited by 10
  21. 2021GCN-SL: Graph Convolutional Networks with Structure Learning for Graphs under Heterophilycited by 5
  22. 2020Parallel and distributed asynchronous adaptive stochastic gradient methodscited by 4
  23. 2021On the Importance of Sampling in Training GCNs: Tighter Analysis and Variance Reductioncited by 4
  24. 2021Label-GCN: An Effective Method for Adding Label Propagation to Graph Convolutional Networkscited by 2
  25. 2024Self-Attention Empowered Graph Convolutional Network for Structure Learning and Node Embeddingcited by 2
  26. 2020MG-GCN: Fast and Effective Learning with Mix-grained Aggregators for Training Large Graph Convolutional Networkscited by 1
  27. 2024PromptGCN: Bridging Subgraph Gaps in Lightweight GCNscited by 1

Built on by

Not cited yet.