mlsanityData to 14 Oct 2026

GraphFM: Improving Large-Scale GNN Training via Feature Momentum

arXiv 2206.07161 ↗First posted 14 Jun 2022cs.LG cs.AIcited by 31

Not on the map. Why?

Abstract (arXiv)

Training of graph neural networks (GNNs) for large-scale node classification is challenging. A key difficulty lies in obtaining accurate hidden node representations while avoiding the neighborhood explosion problem. Here, we propose a new technique, named feature momentum (FM), that uses a momentum step to incorporate historical embeddings when updating feature representations. We develop two specific algorithms, known as GraphFM-IB and GraphFM-OB, that consider in-batch and out-of-batch data, respectively. GraphFM-IB applies FM to in-batch sampled data, while GraphFM-OB applies FM to out-of-batch data that are 1-hop neighborhood of in-batch data. We provide a convergence analysis for GraphFM-IB and some theoretical insight for GraphFM-OB. Empirically, we observe that GraphFM-IB can effectively alleviate the neighborhood explosion problem of existing methods. In addition, GraphFM-OB achieves promising performance on multiple large-scale graph datasets.

Key facts. Cited by 31 papers in the map (data through 14 Oct 2026). Builds on 37 of its 44 references that are in the map. First posted on arXiv 14 Jun 2022.

Builds on

37 of 44 references.

References · 37

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  3. 2017Graph Attention Networkscited by 8,923
  4. 2017Inductive Representation Learning on Large Graphscited by 6,320
  5. 2018How Powerful are Graph Neural Networks?cited by 4,393
  6. 2017Neural Message Passing for Quantum Chemistrycited by 3,487
  7. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,434
  8. 2019Simplifying Graph Convolutional Networkscited by 1,477
  9. 2020Simple and Deep Graph Convolutional Networkscited by 860
  10. 2018Link Prediction Based on Graph Neural Networkscited by 832
  11. 2019DropEdge: Towards Deep Graph Convolutional Networks on Node Classificationcited by 680
  12. 2019GraphSAINT: Graph Sampling Based Inductive Learning Methodcited by 618
25 more
  1. 2018FastGCN: Fast Learning with Graph Convolutional Networks via Importance Samplingcited by 584
  2. 2019DeepGCNs: Can GCNs Go as Deep as CNNs?cited by 527
  3. 2019Graph U-Netscited by 492
  4. 2020Principal Neighbourhood Aggregation for Graph Netscited by 478
  5. 2021OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphscited by 329
  6. 2020Towards Deeper Graph Neural Networkscited by 296
  7. 2020SIGN: Scalable Inception Graph Neural Networkscited by 284
  8. 2020Combining Label Propagation and Simple Models Out-performs Graph Neural Networkscited by 202
  9. 2018Adaptive Sampling Towards Fast Graph Representation Learningcited by 180
  10. 2019Layer-Dependent Importance Sampling for Training Deep and Large Graph Convolutional Networkscited by 177
  11. 2020Scaling Graph Neural Networks with Approximate PageRankcited by 174
  12. 2021Training Graph Neural Networks with 1000 Layerscited by 169
  13. 2021Spherical Message Passing for 3D Graph Networkscited by 168
  14. 2018Large-Scale Learnable Graph Convolutional Networkscited by 113
  15. 2022Decoupling the Depth and Scope of Graph Neural Networkscited by 94
  16. 2021GNNAutoScale: Scalable and Expressive Graph Neural Networks via Historical Embeddingscited by 90
  17. 2020Scalable Graph Neural Networks via Bidirectional Propagationcited by 87
  18. 2021DIG: A Turnkey Library for Diving into Graph Deep Learning Researchcited by 76
  19. 2020Minimal Variance Sampling with Provable Guarantees for Fast Training of Graph Neural Networkscited by 54
  20. 2021A Novel Convergence Analysis for Algorithms of the Adam Familycited by 50
  21. 2020L2-GCN: Layer-Wise and Learned Efficient Training of Graph Convolutional Networkscited by 48
  22. 2022PipeGCN: Efficient Full-Graph Training of Graph Convolutional Networks with Pipelined Feature Communicationcited by 45
  23. 2020Topology-Aware Graph Pooling Networkscited by 44
  24. 2020Stochastic Multi-level Composition Optimization Algorithms with Level-Independent Convergence Ratescited by 25
  25. 2022Neighbor2Seq: Deep Learning on Massive Graphs by Transforming Neighbors to Sequencescited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 15; 2024: 7; 2025: 3; 2026: 4.

31 papers.

Sort:

By area:

  1. 2022Learning Hierarchical Protein Representations via Complete 3D Graph Networkscited by 55
  2. 2023Does Graph Distillation See Like Vision Dataset Counterpart?cited by 45
  3. 2024Complete and Efficient Graph Transformers for Crystal Material Property Predictioncited by 34
  4. 2022MLPInit: Embarrassingly Simple GNN Training Acceleration with MLP Initializationcited by 31
  5. 2024Linear-Time Graph Neural Networks for Scalable Recommendationscited by 31
  6. 2023LMC: Fast Training of GNNs via Subgraph Sampling with Provable Convergencecited by 26
  7. 2024Graph Neural Networks for Financial Fraud Detection: A Reviewcited by 25
  8. 2023A Survey on Graph Neural Network Acceleration: Algorithms, Systems, and Customized Hardwarecited by 18
  9. 2023LazyGNN: Large-Scale Graph Neural Networks via Lazy Propagationcited by 16
  10. 2024Graph Neural Networks for Tabular Data Learning: A Survey with Taxonomy and Directionscited by 13
  11. 2025DeepGate4: Efficient and Effective Representation Learning for Circuit Design at Scalecited by 12
  12. 2023Learning Large Graph Property Prediction via Graph Segment Trainingcited by 9
19 more
  1. 2024Two Trades is not Baffled: Condensing Graph via Crafting Rational Gradient Matchingcited by 8
  2. 2023Label Deconvolution for Node Representation Learning on Large-scale Attributed Graphs against Learning Biascited by 7
  3. 2023GRAPES: Learning to Sample Graphs for Scalable Graph Neural Networkscited by 7
  4. 2023Staleness-Alleviated Distributed GNN Training via Online Dynamic-Embedding Predictioncited by 6
  5. 2025Accurate and Scalable Graph Neural Networks via Message Invariancecited by 3
  6. 2023Decentralized Multi-Level Compositional Optimization Algorithms with Level-Independent Convergence Ratecited by 2
  7. 2023A Local Graph Limits Perspective on Sampling-Based GNNscited by 2
  8. 2023MuseGNN: Forming Scalable, Convergent GNN Layers that Minimize a Sampling-Based Energycited by 2
  9. 2024Haste Makes Waste: A Simple Approach for Scaling Graph Neural Networkscited by 2
  10. 2024PipeNet: Question Answering with Semantic Pruning over Knowledge Graphscited by 1
  11. 2026Rethinking Sales Lead Scoring with LLM-based Hierarchical Preference Rankingcited by 1
  12. 2023Asymmetric Learning for Graph Neural Network based Link Predictioncited by 0
  13. 2023Provably Convergent Subgraph-wise Sampling for Fast GNN Trainingcited by 0
  14. 2023SKU-Patch: Towards Efficient Instance Segmentation for Unseen Objects in Auto-Storecited by 0
  15. 2023Synergistic Signals: Exploiting Co-Engagement and Semantic Links via Graph Neural Networkscited by 0
  16. 2025VISAGNN: Versatile Staleness-Aware Efficient Training on Large-Scale Graphscited by 0
  17. 2026Ftextsuperscript2LP-AP: Fast & Flexible Label Propagation with Adaptive Propagation Kernelcited by 0
  18. 2026Protein Representation Learning with Secondary-Structure and Energy-Filtered Hydrogen-Bond Graphscited by 0
  19. 2026SNI-GNN: SmartNIC-Assisted Full-Graph GNN Training with In-Network Embedding Predictioncited by 0