# GraphACT: Accelerating GCN Training on CPU-FPGA Heterogeneous Platforms

arXiv: https://arxiv.org/abs/2001.02498 · HTML: https://mlsanity.com/paper/2001.02498/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Dec 2019.

## Built on by (most cited first, 30 of 31)

- [Sampling methods for efficient training of graph convolutional networks: A survey](https://mlsanity.com/paper/2103.05872/) (2021)
- [Distributed Graph Neural Network Training: A Survey](https://mlsanity.com/paper/2211.00216/) (2022)
- [A Comprehensive Survey of Dynamic Graph Neural Networks: Models, Frameworks, Benchmarks, Experiments and Challenges](https://mlsanity.com/paper/2405.00476/) (2024)
- [Parallel and Distributed Graph Neural Networks: An In-Depth Concurrency Analysis](https://mlsanity.com/paper/2205.09702/) (2022)
- [I-GCN: A Graph Convolutional Network Accelerator with Runtime Locality Enhancement through Islandization](https://mlsanity.com/paper/2203.03606/) (2022)
- [QGTC: Accelerating Quantized Graph Neural Networks via GPU Tensor Core](https://mlsanity.com/paper/2111.09547/) (2021)
- [A Survey on Graph Neural Network Acceleration: Algorithms, Systems, and Customized Hardware](https://mlsanity.com/paper/2306.14052/) (2023)
- [Graph Neural Networks for Charged Particle Tracking on FPGAs](https://mlsanity.com/paper/2112.02048/) (2021)
- [GCoD: Graph Convolutional Network Acceleration via Dedicated Algorithm and Accelerator Co-Design](https://mlsanity.com/paper/2112.11594/) (2021)
- [SmartSAGE: Training Large-scale Graph Neural Networks using In-Storage Processing Architectures](https://mlsanity.com/paper/2205.04711/) (2022)
- [Model-Architecture Co-Design for High Performance Temporal GNN Inference on FPGA](https://mlsanity.com/paper/2203.05095/) (2022)
- [Graph Neural Networks in Particle Physics: Implementations, Innovations, and Challenges](https://mlsanity.com/paper/2203.12852/) (2022)
- [LL-GNN: Low Latency Graph Neural Networks on FPGAs for High Energy Physics](https://mlsanity.com/paper/2209.14065/) (2022)
- [FlowGNN: A Dataflow Architecture for Real-Time Workload-Agnostic Graph Neural Network Inference](https://mlsanity.com/paper/2204.13103/) (2022)
- [Accurate, Low-latency, Efficient SAR Automatic Target Recognition on FPGA](https://mlsanity.com/paper/2301.01454/) (2023)
- [GenGNN: A Generic FPGA Framework for Graph Neural Network Acceleration](https://mlsanity.com/paper/2201.08475/) (2022)
- [Is Machine Learning Ready for Traffic Engineering Optimization?](https://mlsanity.com/paper/2109.01445/) (2021)
- [GNNear: Accelerating Full-Batch Training of Graph Neural Networks with Near-Memory Processing](https://mlsanity.com/paper/2111.00680/) (2021)
- [GROW: A Row-Stationary Sparse-Dense GEMM Accelerator for Memory-Efficient Graph Convolutional Neural Networks](https://mlsanity.com/paper/2203.00158/) (2022)
- [MEGA: A Memory-Efficient GNN Accelerator Exploiting Degree-Aware Mixed-Precision Quantization](https://mlsanity.com/paper/2311.09775/) (2023)
- [VersaGNN: a Versatile accelerator for Graph neural networks](https://mlsanity.com/paper/2105.01280/) (2021)
- [HDReason: Algorithm-Hardware Codesign for Hyperdimensional Knowledge Graph Reasoning](https://mlsanity.com/paper/2403.05763/) (2024)
- [Bottleneck Analysis of Dynamic Graph Neural Network Inference on CPU and GPU](https://mlsanity.com/paper/2210.03900/) (2022)
- [Programmable FPGA-based Memory Controller](https://mlsanity.com/paper/2108.09601/) (2021)
- [H-GCN: A Graph Convolutional Network Accelerator on Versal ACAP Architecture](https://mlsanity.com/paper/2206.13734/) (2022)
- [GNNBuilder: An Automated Framework for Generic Graph Neural Network Accelerator Generation, Simulation, and Optimization](https://mlsanity.com/paper/2303.16459/) (2023)
- [From the Pursuit of Universal AGI Architecture to Systematic Approach to Heterogenous AGI: Addressing Alignment, Energy, & AGI Grand Challenges](https://mlsanity.com/paper/2310.15274/) (2023)
- [SPA-GCN: Efficient and Flexible GCN Accelerator with an Application for Graph Similarity Computation](https://mlsanity.com/paper/2111.05936/) (2021)
- [Accelerating Generic Graph Neural Networks via Architecture, Compiler, Partition Method Co-Design](https://mlsanity.com/paper/2308.08174/) (2023)
- [SiHGNN: Leveraging Properties of Semantic Graphs for Efficient HGNN Acceleration](https://mlsanity.com/paper/2408.15089/) (2024)

## Abstract (arXiv)

Graph Convolutional Networks (GCNs) have emerged as the state-of-the-art deep learning model for representation learning on graphs. It is challenging to accelerate training of GCNs, due to (1) substantial and irregular data communication to propagate information within the graph, and (2) intensive computation to propagate information along the neural network layers. To address these challenges, we design a novel accelerator for training GCNs on CPU-FPGA heterogeneous systems, by incorporating multiple algorithm-architecture co-optimizations. We first analyze the computation and communication characteristics of various GCN training algorithms, and select a subgraph-based algorithm that is well suited for hardware execution. To optimize the feature propagation within subgraphs, we propose a lightweight pre-processing step based on a graph theoretic approach. Such pre-processing performed on the CPU significantly reduces the memory access requirements and the computation to be performed on the FPGA. To accelerate the weight update in GCN layers, we propose a systolic array based design for efficient parallelization. We integrate the above optimizations into a complete hardware pipeline, and analyze its load-balance and resource utilization by accurate performance modeling. We evaluate our design on a Xilinx Alveo U200 board hosted by a 40-core Xeon server. On three large graphs, we achieve an order of magnitude training speedup with negligible accuracy loss, compared with state-of-the-art implementation on a multi-core platform.