Characterizing and Understanding HGNN Training on GPUs
Not on the map. Why?
Abstract (arXiv)
Owing to their remarkable representation capabilities for heterogeneous graph data, Heterogeneous Graph Neural Networks (HGNNs) have been widely adopted in many critical real-world domains such as recommendation systems and medical analysis. Prior to their practical application, identifying the optimal HGNN model parameters tailored to specific tasks through extensive training is a time-consuming and costly process. To enhance the efficiency of HGNN training, it is essential to characterize and analyze the execution semantics and patterns within the training process to identify performance bottlenecks. In this study, we conduct an in-depth quantification and analysis of two mainstream HGNN training scenarios, including single-GPU and multi-GPU distributed training. Based on the characterization results, we disclose the performance bottlenecks and their underlying causes in different HGNN training scenarios and provide optimization guidelines from both software and hardware perspectives.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 29 of its 51 references that are in the map. First posted on arXiv 16 Jul 2024.
Builds on
29 of 51 references.
References · 29
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
- 2017Graph Attention Networkscited by 8,923
- 2017Inductive Representation Learning on Large Graphscited by 6,320
- 2018How Powerful are Graph Neural Networks?cited by 4,393
- 2019A Comprehensive Survey on Graph Neural Networkscited by 2,919
- 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,434
- 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,694
- 2018Graph Neural Networks: A Review of Methods and Applicationscited by 1,576
- 2019Heterogeneous Graph Attention Networkcited by 734
- 2020Graph Neural Networks in Recommender Systems: A Surveycited by 494
- 2020MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embeddingcited by 304
- 2019Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networkscited by 242
17 more
- 2021Are we really making much progress? Revisiting, benchmarking, and refining heterogeneous graph neural networkscited by 176
- 2020A Survey on Heterogeneous Graph Embedding: Methods, Techniques, Applications and Sourcescited by 126
- 2015A Survey of Heterogeneous Information Network Analysiscited by 110
- 2020Relational Graph Attention Network for Aspect-based Sentiment Analysiscited by 104
- 2021Dorylus: Affordable, Scalable, and Accurate GNN Training with Distributed CPU Servers and Serverless Threadscited by 51
- 2021Sampling methods for efficient training of graph convolutional networks: A surveycited by 40
- 2021Distributed Hybrid CPU and GPU training for Graph Neural Networks on Billion-Scale Graphscited by 33
- 2021Understanding GNN Computational Graph: A Coordinated Computation, IO, and Memory Perspectivecited by 28
- 2020Characterizing and Understanding GCNs on GPUcited by 22
- 2020Architectural Implications of Graph Neural Networkscited by 12
- 2022Characterizing and Understanding Distributed GNN Training on GPUscited by 7
- 2024GDR-HGNN: A Heterogeneous Graph Neural Networks Accelerator Frontend with Graph Decoupling and Recouplingcited by 4
- 2022Characterizing and Understanding HGNNs on GPUscited by 3
- 2022Characterizing the Efficiency of Graph Neural Network Frameworks with a Magnifying Glasscited by 3
- 2023HiHGNN: Accelerating HGNNs through Parallelism and Data Reusability Exploitationcited by 3
- 2023A Survey of Graph Pre-processing Methods: From Algorithmic to Hardware Perspectivescited by 2
- 2024ADE-HGNN: Accelerating HGNNs through Attention Disparity Exploitationcited by 2
Built on by
Citing papers per year: 2024: 1.
1 paper.