# Low-latency Mini-batch GNN Inference on CPU-FPGA Heterogeneous Platform

arXiv: https://arxiv.org/abs/2206.08536 · HTML: https://mlsanity.com/paper/2206.08536/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jun 2022.

## Built on by (most cited first, 7 of 7)

- [MaxK-GNN: Extremely Fast GPU Kernel Design for Accelerating Graph Neural Networks Training](https://mlsanity.com/paper/2312.08656/) (2023)
- [MaGNAS: A Mapping-Aware Graph Neural Architecture Search Framework for Heterogeneous MPSoC Deployment](https://mlsanity.com/paper/2307.08065/) (2023)
- [Bottleneck Analysis of Dynamic Graph Neural Network Inference on CPU and GPU](https://mlsanity.com/paper/2210.03900/) (2022)
- [Exploiting On-chip Heterogeneity of Versal Architecture for GNN Inference Acceleration](https://mlsanity.com/paper/2308.02749/) (2023)
- [Research on Low-Latency Inference and Training Efficiency Optimization for Graph Neural Network and Large Language Model-Based Recommendation Systems](https://mlsanity.com/paper/2507.01035/) (2025)
- [Graph-Powered Defense: Controller Area Network Intrusion Detection for Unmanned Aerial Vehicles](https://mlsanity.com/paper/2412.02539/) (2024)
- [Bit-Flip Fault Attack: Crushing Graph Neural Networks via Gradual Bit Search](https://mlsanity.com/paper/2507.05531/) (2025)

## Abstract (arXiv)

Mini-batch inference of Graph Neural Networks (GNNs) is a key problem in many real-world applications. Recently, a GNN design principle of model depth-receptive field decoupling has been proposed to address the well-known issue of neighborhood explosion. Decoupled GNN models achieve higher accuracy than original models and demonstrate excellent scalability for mini-batch inference. We map Decoupled GNNs onto CPU-FPGA heterogeneous platforms to achieve low-latency mini-batch inference. On the FPGA platform, we design a novel GNN hardware accelerator with an adaptive datapath denoted Adaptive Computation Kernel (ACK) that can execute various computation kernels of GNNs with low-latency: (1) for dense computation kernels expressed as matrix multiplication, ACK works as a systolic array with fully localized connections, (2) for sparse computation kernels, ACK follows the scatter-gather paradigm and works as multiple parallel pipelines to support the irregular connectivity of graphs. The proposed task scheduling hides the CPU-FPGA data communication overhead to reduce the inference latency. We develop a fast design space exploration algorithm to generate a single accelerator for multiple target GNN models. We implement our accelerator on a state-of-the-art CPU-FPGA platform and evaluate the performance using three representative models (GCN, GraphSAGE, and GAT). Results show that our CPU-FPGA implementation achieves 21.4-50.8×, 2.9-21.6×, 4.7× latency reduction compared with state-of-the-art implementations on CPU-only, CPU-GPU and CPU-FPGA platforms.