DGC: Training Dynamic Graphs with Spatio-Temporal Non-Uniformity using Graph Partitioning by Chunks
Not on the map. Why?
Abstract (arXiv)
Dynamic Graph Neural Network (DGNN) has shown a strong capability of learning dynamic graphs by exploiting both spatial and temporal features. Although DGNN has recently received considerable attention by AI community and various DGNN models have been proposed, building a distributed system for efficient DGNN training is still challenging. It has been well recognized that how to partition the dynamic graph and assign workloads to multiple GPUs plays a critical role in training acceleration. Existing works partition a dynamic graph into snapshots or temporal sequences, which only work well when the graph has uniform spatio-temporal structures. However, dynamic graphs in practice are not uniformly structured, with some snapshots being very dense while others are sparse. To address this issue, we propose DGC, a distributed DGNN training system that achieves a 1.25x - 7.52x speedup over the state-of-the-art in our testbed. DGC's success stems from a new graph partitioning method that partitions dynamic graphs into chunks, which are essentially subgraphs with modest training workloads and few inter connections. This partitioning algorithm is based on graph coarsening, which can run very fast on large graphs. In addition, DGC has a highly efficient run-time, powered by the proposed chunk fusion and adaptive stale aggregation techniques. Extensive experimental results on 3 typical DGNN models and 4 popular dynamic graph datasets are presented to show the effectiveness of DGC.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 39 of its 80 references that are in the map. First posted on arXiv 7 Sep 2023.
Builds on
39 of 80 references.
References · 39
- 2017Attention Is All You Needcited by 59,932
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
- 2017Graph Attention Networkscited by 8,854
- 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
- 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
- 2019Simplifying Graph Convolutional Networkscited by 1,470
- 2020LightGCN: Simplifying and Powering Graph Convolution Network for Recommendationcited by 1,134
- 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,133
- 2019GraphSAINT: Graph Sampling Based Inductive Learning Methodcited by 618
- 2019Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networkscited by 540
27 more
- 2019EvolveGCN: Evolving Graph Convolutional Networks for Dynamic Graphscited by 443
- 2018T-GCN: A Temporal Graph ConvolutionalNetwork for Traffic Predictioncited by 399
- 2019Layer-Dependent Importance Sampling for Training Deep and Large Graph Convolutional Networkscited by 177
- 2018GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphscited by 164
- 2019AliGraph: A Comprehensive Graph Neural Network Platformcited by 152
- 2020DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphscited by 139
- 2021PyTorch Geometric Temporal: Spatiotemporal Signal Processing with Neural Machine Learning Modelscited by 118
- 2018Streaming Graph Neural Networkscited by 91
- 2021Scaling Up Graph Neural Networks Via Graph Coarseningcited by 80
- 2020PinnerSage: Multi-Modal User Embedding Framework for Recommendations at Pinterestcited by 75
- 2021DistGNN: Scalable Distributed Training for Large-Scale Graph Neural Networkscited by 59
- 2022TGL: A General Framework for Temporal GNN Training on Billion-Scale Graphscited by 59
- 2020GNNAdvisor: An Adaptive and Efficient Runtime System for GNN Acceleration on GPUscited by 51
- 2021Dorylus: Affordable, Scalable, and Accurate GNN Training with Distributed CPU Servers and Serverless Threadscited by 51
- 2020Reducing Communication in Graph Neural Network Trainingcited by 50
- 2021BGL: GPU-Efficient GNN Training by Optimizing Graph Data I/O and Preprocessingcited by 50
- 2020L2-GCN: Layer-Wise and Learned Efficient Training of Graph Convolutional Networkscited by 48
- 2021Graph-Free Knowledge Distillation for Graph Neural Networkscited by 46
- 2022PipeGCN: Efficient Full-Graph Training of Graph Convolutional Networks with Pipelined Feature Communicationcited by 45
- 2021FedGraph: Federated Graph Learning with Intelligent Samplingcited by 34
- 2020Binary Graph Neural Networkscited by 32
- 2019A Unifying Framework for Spectrum-Preserving Graph Sparsification and Coarseningcited by 31
- 2021HET: Scaling out Huge Embedding Model Training via Cache-enabled Distributed Frameworkcited by 28
- 2020Bi-GCN: Binary Graph Convolutional Networkcited by 27
- 2021QGTC: Accelerating Quantized Graph Neural Networks via GPU Tensor Corecited by 18
- 2021Efficient Scaling of Dynamic Graph Neural Networkscited by 11
- 2023PiPAD: Pipelined and Parallel Dynamic GNN Training on GPUscited by 4
Built on by
Citing papers per year: 2024: 1.
1 paper.