mlsanityData to 7 Oct 2026

Learning by Transference: Training Graph Neural Networks on Growing Graphs

arXiv 2106.03693 ↗First posted 7 Jun 2021cs.LG eess.SPcited by 22

Not on the map. Why?

Abstract (arXiv)

Graph neural networks (GNNs) use graph convolutions to exploit network invariances and learn meaningful feature representations from network data. However, on large-scale graphs convolutions incur in high computational cost, leading to scalability limitations. Leveraging the graphon -- the limit object of a graph -- in this paper we consider the problem of learning a graphon neural network (WNN) -- the limit object of a GNN -- by training GNNs on graphs sampled from the graphon. Under smoothness conditions, we show that: (i) the expected distance between the learning steps on the GNN and on the WNN decreases asymptotically with the size of the graph, and (ii) when training on a sequence of growing graphs, gradient descent follows the learning direction of the WNN. Inspired by these results, we propose a novel algorithm to learn GNNs on large-scale graphs that, starting from a moderate number of nodes, successively increases the size of the graph during training. This algorithm is further benchmarked on a decentralized control problem, where it retains comparable performance to its large-scale counterpart at a reduced computational cost.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 38 of its 65 references that are in the map. First posted on arXiv 7 Jun 2021.

Builds on

38 of 65 references.

References · 38

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  3. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  4. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
  5. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,085
  6. 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,684
  7. 2018Graph Neural Networks: A Review of Methods and Applicationscited by 1,571
  8. 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,133
  9. 2013Spectral Networks and Locally Connected Networks on Graphscited by 1,097
  10. 2015Gated Graph Sequence Neural Networkscited by 792
  11. 2015Convolutional Networks on Graphs for Learning Molecular Fingerprintscited by 751
  12. 2019Graph Neural Networks for Social Recommendationcited by 711
26 more
  1. 2018Gradient Descent Finds Global Minima of Deep Neural Networkscited by 572
  2. 2018Lipschitz regularity of deep neural networks: analysis and efficient estimationcited by 413
  3. 2018Invariant and Equivariant Graph Networkscited by 339
  4. 2019Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networkscited by 290
  5. 2020Can Graph Neural Networks Count Substructures?cited by 235
  6. 2019Universal Invariant and Equivariant Graph Neural Networkscited by 182
  7. 2018Learning Human-Object Interactions by Graph Parsing Neural Networkscited by 176
  8. 2019On the equivalence between graph isomorphism testing and function approximation with GNNscited by 172
  9. 2020Machine Learning on Graphs: A Model and Comprehensive Taxonomycited by 156
  10. 2019Stability Properties of Graph Neural Networkscited by 125
  11. 2020Graphon Neural Networks and the Transferability of Graph Neural Networkscited by 117
  12. 2020Exactly Computing the Local Lipschitz Constant of ReLU Networkscited by 102
  13. 2018Convolutional Neural Network Architectures for Signals Supported on Graphscited by 94
  14. 2019Transferability of Spectral Graph Convolutional Neural Networkscited by 90
  15. 2020Lipschitz constant estimation of Neural Networks via sparse polynomial optimizationcited by 87
  16. 2019Graph Neural Networks for Decentralized Multi-Robot Path Planningcited by 84
  17. 2019GMNN: Graph Markov Neural Networkscited by 82
  18. 2020Convergence and Stability of Graph Convolutional Networks on Large Random Graphscited by 75
  19. 2019Learning Decentralized Controllers for Robot Swarms with Graph Neural Networkscited by 62
  20. 2020Learning to Hash with Graph Neural Networks for Recommender Systemscited by 31
  21. 2020Approximating Lipschitz continuous functions with GroupSort neural networkscited by 27
  22. 2019Invariance-Preserving Localized Activation Functions for Graph Neural Networkscited by 14
  23. 2021Distributed Linear-Quadratic Control with Graph Neural Networkscited by 12
  24. 2019The Graphon Fourier Transformcited by 10
  25. 2017Graphons: A Nonparametric Method to Model, Estimate, and Design Algorithms for Massive Networkscited by 9
  26. 2019Gated Graph Convolutional Recurrent Neural Networkscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 5; 2024: 6; 2025: 3; 2026: 5.

22 papers.

Sort:

  1. 2022Generalization Analysis of Message Passing Neural Networks on Large Random Graphscited by 58
  2. 2023Graph Neural Tangent Kernel: Convergence on Large Graphscited by 16
  3. 2024Generalization Bounds for Message Passing Networks on Mixture of Graphonscited by 16
  4. 2024A Manifold Perspective on the Statistical Generalization of Graph Neural Networkscited by 16
  5. 2023Convergence of Message Passing Graph Neural Networks with Generic Aggregation On Large Random Graphscited by 9
  6. 2024Generalization of Geometric Graph Neural Networks with Lipschitz Loss Functionscited by 6
  7. 2022Training Graph Neural Networks on Growing Stochastic Graphscited by 5
  8. 2024Generalization of Graph Neural Networks is Robust to Model Mismatchcited by 4
  9. 2022Unveiling the Sampling Density in Non-Uniform Geometric Graphscited by 2
  10. 2023Transferability of Graph Neural Networks using Graphon and Sampling Theoriescited by 2
  11. 2023A Local Graph Limits Perspective on Sampling-Based GNNscited by 2
  12. 2024Online Learning Of Expanding Graphscited by 2
10 more
  1. 2023A Poincar'e Inequality and Consistency Results for Signal Sampling on Large Graphscited by 1
  2. 2026Stability of Flow Models for Graph Signalscited by 1
  3. 2024Distributed Training of Large Graph Neural Networks with Variable Communication Ratescited by 0
  4. 2025Graph Semi-Supervised Learning for Point Classification on Data Manifoldscited by 0
  5. 2025Unrolling Dynamic Programming via Graph Filterscited by 0
  6. 2025Adaptive Graph Coarsening for Efficient GNN Trainingcited by 0
  7. 2026Size Transferability of Graph Transformers with Convolutional Positional Encodingscited by 0
  8. 2026Stochastic Sequential Decision Making over Expanding Networks with Graph Filteringcited by 0
  9. 2026A Topological Characterization of Graph Neural Networks via Stochastic Block Model Embeddings on the n-Spherecited by 0
  10. 2026Attention Graphons: A Graph Limit Perspective on Graph Transformerscited by 0