mlsanityData to 7 Oct 2026

IGLU: Efficient GCN Training via Lazy Updates

arXiv 2109.13995 ↗First posted 28 Sep 2021cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Training multi-layer Graph Convolution Networks (GCN) using standard SGD techniques scales poorly as each descent step ends up updating node embeddings for a large portion of the graph. Recent attempts to remedy this sub-sample the graph that reduces compute but introduce additional variance and may offer suboptimal performance. This paper develops the IGLU method that caches intermediate computations at various GCN layers thus enabling lazy updates that significantly reduce the compute cost of descent. IGLU introduces bounded bias into the gradients but nevertheless converges to a first-order saddle point under standard assumptions such as objective smoothness. Benchmark experiments show that IGLU offers up to 1.2% better accuracy despite requiring up to 88% less compute.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 27 of its 30 references that are in the map. First posted on arXiv 28 Sep 2021.

Builds on

27 of 30 references.

References · 27

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  4. 2017Graph Attention Networkscited by 8,854
  5. 2017Inductive Representation Learning on Large Graphscited by 6,284
  6. 2018How Powerful are Graph Neural Networks?cited by 4,355
  7. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  8. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  9. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,085
  10. 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,133
  11. 2013Spectral Networks and Locally Connected Networks on Graphscited by 1,097
  12. 2018Predict then Propagate: Graph Neural Networks meet Personalized PageRankcited by 987
15 more
  1. 2020Simple and Deep Graph Convolutional Networkscited by 857
  2. 2020Geom-GCN: Geometric Graph Convolutional Networkscited by 821
  3. 2019GraphSAINT: Graph Sampling Based Inductive Learning Methodcited by 618
  4. 2018FastGCN: Fast Learning with Graph Convolutional Networks via Importance Samplingcited by 583
  5. 2019Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networkscited by 540
  6. 2020Principal Neighbourhood Aggregation for Graph Netscited by 477
  7. 2020SIGN: Scalable Inception Graph Neural Networkscited by 283
  8. 2018Adaptive Sampling Towards Fast Graph Representation Learningcited by 180
  9. 2019Layer-Dependent Importance Sampling for Training Deep and Large Graph Convolutional Networkscited by 177
  10. 2020Scaling Graph Neural Networks with Approximate PageRankcited by 174
  11. 2020Graph-based, Self-Supervised Program Repair from Diagnostic Feedbackcited by 93
  12. 2021GNNAutoScale: Scalable and Expressive Graph Neural Networks via Historical Embeddingscited by 90
  13. 2020Minimal Variance Sampling with Provable Guarantees for Fast Training of Graph Neural Networkscited by 54
  14. 2020L2-GCN: Layer-Wise and Learned Efficient Training of Graph Convolutional Networkscited by 48
  15. 2020Bandit Samplers for Training Graph Neural Networkscited by 30

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 1; 2024: 1; 2025: 2.

7 papers.

  1. 2022Transformers from an Optimization Perspectivecited by 27
  2. 2022RSC: Accelerating Graph Neural Networks Training via Randomized Sparse Computationscited by 12
  3. 2025Fast RoPE Attention: Combining the Polynomial Method and Fast Fourier Transformcited by 9
  4. 2023Simplifying Distributed Neural Network Training on Massive Graphs: Randomized Partitions Improve Model Aggregationcited by 7
  5. 2022Graph Generative Model for Benchmarking Graph Neural Networkscited by 6
  6. 2024SORSA: Singular Values and Orthonormal Regularized Singular Vectors Adaptation of Large Language Modelscited by 5
  7. 2025Nonparametric Teaching for Graph Property Learnerscited by 5