mlsanityData to 7 Oct 2026

The Evolution of Distributed Systems for Graph Neural Networks and their Origin in Graph Processing and Deep Learning: A Survey

arXiv 2305.13854 ↗First posted 23 May 2023cs.DC cs.LGcited by 13

Not on the map. Why?

Abstract (arXiv)

Graph Neural Networks (GNNs) are an emerging research field. This specialized Deep Neural Network (DNN) architecture is capable of processing graph structured data and bridges the gap between graph processing and Deep Learning (DL). As graphs are everywhere, GNNs can be applied to various domains including recommendation systems, computer vision, natural language processing, biology and chemistry. With the rapid growing size of real world graphs, the need for efficient and scalable GNN training solutions has come. Consequently, many works proposing GNN systems have emerged throughout the past few years. However, there is an acute lack of overview, categorization and comparison of such systems. We aim to fill this gap by summarizing and categorizing important methods and techniques for large-scale GNN solutions. In addition, we establish connections between GNN systems, graph processing systems and DL systems.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 99 of its 225 references that are in the map. First posted on arXiv 23 May 2023.

Builds on

99 of 225 references.

References · 99

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  4. 2017Graph Attention Networkscited by 8,854
  5. 2017Inductive Representation Learning on Large Graphscited by 6,284
  6. 2018How Powerful are Graph Neural Networks?cited by 4,355
  7. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  8. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
  9. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
  10. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  11. 2018Graph Neural Networks: A Review of Methods and Applicationscited by 1,571
  12. 2019Simplifying Graph Convolutional Networkscited by 1,470
38 more
  1. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,437
  2. 2016Variational Graph Auto-Encoderscited by 1,373
  3. 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,133
  4. 2016An overview of gradient descent optimization algorithmscited by 1,021
  5. 2018Representation Learning on Graphs with Jumping Knowledge Networkscited by 875
  6. 2015Gated Graph Sequence Neural Networkscited by 792
  7. 2019Graph Neural Networks for Social Recommendationcited by 711
  8. 2019GraphSAINT: Graph Sampling Based Inductive Learning Methodcited by 618
  9. 2018FastGCN: Fast Learning with Graph Convolutional Networks via Importance Samplingcited by 583
  10. 2019Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networkscited by 540
  11. 2020Temporal Graph Networks for Deep Learning on Dynamic Graphscited by 521
  12. 2018Hypergraph Neural Networkscited by 482
  13. 2017Representation Learning on Graphs: Methods and Applicationscited by 456
  14. 2019Predicting Dynamic Embedding Trajectory in Temporal Interaction Networkscited by 443
  15. 2018Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Networkcited by 415
  16. 2018Deep Learning on Graphs: A Surveycited by 393
  17. 2016Learning Multiagent Communication with Backpropagationcited by 355
  18. 2018Graph Convolutional Networks for Text Classificationcited by 345
  19. 2017Residual Gated Graph ConvNetscited by 341
  20. 2021OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphscited by 328
  21. 2020MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embeddingcited by 304
  22. 2020Towards Deeper Graph Neural Networkscited by 296
  23. 2020DeeperGCN: All You Need to Train Deeper GCNscited by 249
  24. 2019Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networkscited by 241
  25. 2019Position-aware Graph Neural Networkscited by 203
  26. 2015MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systemscited by 195
  27. 2019Layer-Dependent Importance Sampling for Training Deep and Large Graph Convolutional Networkscited by 177
  28. 2020A Survey on Knowledge Graph-Based Recommender Systemscited by 170
  29. 2021Training Graph Neural Networks with 1000 Layerscited by 169
  30. 2020Machine Learning on Graphs: A Model and Comprehensive Taxonomycited by 156
  31. 2018Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysiscited by 154
  32. 2019AliGraph: A Comprehensive Graph Neural Network Platformcited by 152
  33. 2021Graph-less Neural Networks: Teaching Old MLPs New Tricks via Distillationcited by 151
  34. 2017Revisiting Distributed Synchronous SGDcited by 142
  35. 2020DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphscited by 139
  36. 2017Predicting multicellular function through multi-layer tissue networkscited by 138
  37. 2017Encoding Sentences with Graph Convolutional Networks for Semantic Role Labelingcited by 137
  38. 2020Degree-Quant: Quantization-Aware Training for Graph Neural Networkscited by 93

50 most cited of 99 All 99 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 3; 2025: 4; 2026: 3.

13 papers.

Sort:

By area:

  1. 2023Comprehensive Evaluation of GNN Training Systems: A Data Management Perspectivecited by 9
  2. 2023An Experimental Comparison of Partitioning Strategies for Distributed Graph Neural Network Trainingcited by 4
  3. 2025Graph Neural Network-Based Collaborative Perception for Adaptive Scheduling in Distributed Systemscited by 4
  4. 2026Forget Many, Forget Right: Scalable and Precise Concept Unlearning in Diffusion Modelscited by 4
  5. 2023Limited Communications Distributed Optimization via Deep Unfolded Distributed ADMMcited by 2
  6. 2024D3-GNN: Dynamic Distributed Dataflow for Streaming Graph Neural Networkscited by 2
  7. 2024Graph Retention Networks for Dynamic Graphscited by 2
  8. 2025AMUN: Adversarial Machine UNlearningcited by 2
  9. 2025WARP: Weight Teleportation for Attack-Resilient Unlearning Protocolscited by 1
  10. 2026Forget-It-All: Multi-Concept Machine Unlearning via Concept-Aware Neuron Maskingcited by 1
  11. 2024Vision Paper: Designing Graph Neural Networks in Compliance with the European Artificial Intelligence Actcited by 0
  12. 2025Fair Minimum Labeling: Efficient Temporal Network Activations for Reachability and Equitycited by 0
1 more
  1. 2026Factored Value Functions for Graph-Based Multi-Agent Reinforcement Learningcited by 0