mlsanityData to 14 Oct 2026

PiPAD: Pipelined and Parallel Dynamic GNN Training on GPUs

arXiv 2301.00391 ↗First posted 1 Jan 2023cs.LG cs.DCcited by 4

Not on the map. Why?

Abstract (arXiv)

Dynamic Graph Neural Networks (DGNNs) have been broadly applied in various real-life applications, such as link prediction and pandemic forecast, to capture both static structural information and temporal characteristics from dynamic graphs. Combining both time-dependent and -independent components, DGNNs manifest substantial parallel computation and data reuse potentials, but suffer from severe memory access inefficiency and data transfer overhead under the canonical one-graph-at-a-time training pattern. To tackle the challenges, we propose PiPAD, a underlinePipelined and underlinePArallel underlineDGNN training framework for the end-to-end performance optimization on GPUs. From both the algorithm and runtime level, PiPAD holistically reconstructs the overall training paradigm from the data organization to computation manner. Capable of processing multiple graph snapshots in parallel, PiPAD eliminates the unnecessary data transmission and alleviates memory access inefficiency to improve the overall performance. Our evaluation across various datasets shows PiPAD achieves 1.22×-9.57× speedup over the state-of-the-art DGNN frameworks on three representative models.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 26 of its 52 references that are in the map. First posted on arXiv 1 Jan 2023.

Builds on

26 of 52 references.

References · 26

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  3. 2017Graph Attention Networkscited by 8,923
  4. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  5. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,434
  6. 2020Adaptive Graph Convolutional Recurrent Network for Traffic Forecastingcited by 530
  7. 2019EvolveGCN: Evolving Graph Convolutional Networks for Dynamic Graphscited by 447
  8. 2018T-GCN: A Temporal Graph ConvolutionalNetwork for Traffic Predictioncited by 401
  9. 2019Representation Learning for Dynamic Graphs: A Surveycited by 265
  10. 2019Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networkscited by 242
  11. 2019AliGraph: A Comprehensive Graph Neural Network Platformcited by 153
  12. 2020Memory-Efficient Pipeline-Parallel DNN Trainingcited by 134
14 more
  1. 2020Foundations and modelling of dynamic networks using Dynamic Graph Neural Networks: A surveycited by 124
  2. 2021PyTorch Geometric Temporal: Spatiotemporal Signal Processing with Neural Machine Learning Modelscited by 118
  3. 2020Computing Graph Neural Networks: A Survey from Algorithms to Acceleratorscited by 81
  4. 2022TGL: A General Framework for Temporal GNN Training on Billion-Scale Graphscited by 61
  5. 2020GNNAdvisor: An Adaptive and Efficient Runtime System for GNN Acceleration on GPUscited by 51
  6. 2021Dorylus: Affordable, Scalable, and Accurate GNN Training with Distributed CPU Servers and Serverless Threadscited by 51
  7. 2019Memory Augmented Graph Neural Networks for Sequential Recommendationcited by 46
  8. 2022PipeGCN: Efficient Full-Graph Training of Graph Convolutional Networks with Pipelined Feature Communicationcited by 45
  9. 2020GE-SpMM: General-purpose Sparse Matrix-Matrix Multiplication on GPUs for Graph Neural Networkscited by 42
  10. 2021Accelerating Training and Inference of Graph Neural Networks with Fast Sampling and Pipeliningcited by 39
  11. 2017Learning Graph Representations with Embedding Propagationcited by 38
  12. 2020Nimble: Lightweight and Parallel GPU Task Scheduling for Deep Learningcited by 17
  13. 2021Efficient Scaling of Dynamic Graph Neural Networkscited by 11
  14. 2014NetworkRepository: An Interactive Data Repository with Multi-scale Visual Analyticscited by 3

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 2.

4 papers.

  1. 2024A Comprehensive Survey of Dynamic Graph Neural Networks: Models, Frameworks, Benchmarks, Experiments and Challengescited by 30
  2. 2024A survey of dynamic graph neural networkscited by 24
  3. 2023Accelerating Sampling and Aggregation Operations in GNN Frameworks with GPU Initiated Direct Storage Accessescited by 6
  4. 2023DGC: Training Dynamic Graphs with Spatio-Temporal Non-Uniformity using Graph Partitioning by Chunkscited by 1