# Exploiting On-chip Heterogeneity of Versal Architecture for GNN Inference Acceleration

arXiv: https://arxiv.org/abs/2308.02749 · HTML: https://mlsanity.com/paper/2308.02749/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Early exit and edge inference, one of 12 directions in Efficient LLM inference and serving.
- Builds on 26 of its 46 references that are in the map.
- First posted on arXiv 4 Aug 2023.

## Placed in

- Efficient LLM inference and serving / [Early exit and edge inference](https://mlsanity.com/inference/early-exit-and-edge-inference/)

## Builds on (most cited first, 26 of 26)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Simplifying Graph Convolutional Networks](https://mlsanity.com/paper/1902.07153/) (2019)
- [Revisiting Semi-Supervised Learning with Graph Embeddings](https://mlsanity.com/paper/1603.08861/) (2016)
- [Link Prediction Based on Graph Neural Networks](https://mlsanity.com/paper/1802.09691/) (2018)
- [GraphSAINT: Graph Sampling Based Inductive Learning Method](https://mlsanity.com/paper/1907.04931/) (2019)
- [Enhancing Graph Neural Network-based Fraud Detectors against Camouflaged Fraudsters](https://mlsanity.com/paper/2008.08692/) (2020)
- [Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networks](https://mlsanity.com/paper/1909.01315/) (2019)
- [A Unified Lottery Ticket Hypothesis for Graph Neural Networks](https://mlsanity.com/paper/2102.06790/) (2021)
- [Meta-GNN: On Few-shot Node Classification in Graph Meta-learning](https://mlsanity.com/paper/1905.09718/) (2019)
- [HyGCN: A GCN Accelerator with Hybrid Architecture](https://mlsanity.com/paper/2001.02514/) (2020)
- [GNNAdvisor: An Adaptive and Efficient Runtime System for GNN Acceleration on GPUs](https://mlsanity.com/paper/2006.06608/) (2020)
- [I-GCN: A Graph Convolutional Network Accelerator with Runtime Locality Enhancement through Islandization](https://mlsanity.com/paper/2203.03606/) (2022)
- [DNNExplorer: A Framework for Modeling and Exploring a Novel Paradigm of FPGA-based DNN Accelerator](https://mlsanity.com/paper/2008.12745/) (2020)
- [Model-Architecture Co-Design for High Performance Temporal GNN Inference on FPGA](https://mlsanity.com/paper/2203.05095/) (2022)
- [HP-GNN: Generating High Throughput GNN Training Implementation on CPU-FPGA Heterogeneous Platform](https://mlsanity.com/paper/2112.11684/) (2021)
- [Accurate, Low-latency, Efficient SAR Automatic Target Recognition on FPGA](https://mlsanity.com/paper/2301.01454/) (2023)
- [HybridDNN: A Framework for High-Performance Hybrid DNN Accelerator Design and Implementation](https://mlsanity.com/paper/2004.03804/) (2020)
- [Low-latency Mini-batch GNN Inference on CPU-FPGA Heterogeneous Platform](https://mlsanity.com/paper/2206.08536/) (2022)
- [GraphAGILE: An FPGA-based Overlay Accelerator for Low-latency GNN Inference](https://mlsanity.com/paper/2302.01769/) (2023)
- [Triple Sparsification of Graph Convolutional Networks without Sacrificing the Accuracy](https://mlsanity.com/paper/2208.03559/) (2022)
- [DYNAMAP: Dynamic Algorithm Mapping Framework for Low Latency CNN Inference](https://mlsanity.com/paper/2012.00912/) (2020)
- [Dynasparse: Accelerating GNN Inference through Dynamic Sparsity Exploitation](https://mlsanity.com/paper/2303.12901/) (2023)
- [H-GCN: A Graph Convolutional Network Accelerator on Versal ACAP Architecture](https://mlsanity.com/paper/2206.13734/) (2022)

## Built on by (most cited first, 2 of 2)

- [BitParticle: Partializing Sparse Dual-Factors to Build Quasi-Synchronizing MAC Arrays for Energy-efficient DNNs](https://mlsanity.com/paper/2507.09780/) (2025)
- [Design Rules for Extreme-Edge Scientific Computing on AI Engines](https://mlsanity.com/paper/2604.19106/) (2026)

## Abstract (arXiv)

Graph Neural Networks (GNNs) have revolutionized many Machine Learning (ML) applications, such as social network analysis, bioinformatics, etc. GNN inference can be accelerated by exploiting data sparsity in the input graph, vertex features, and intermediate data in GNN computations. For dynamic sparsity exploitation, we leverage the heterogeneous computing capabilities of AMD Versal ACAP architecture to accelerate GNN inference. We develop a custom hardware module that executes the sparse primitives of the computation kernel on the Programmable Logic (PL) and efficiently computes the dense primitives using the AI Engine (AIE). To exploit data sparsity during inference, we devise a runtime kernel mapping strategy that dynamically assigns computation tasks to the PL and AIE based on data sparsity. Our implementation on the VCK5000 ACAP platform leads to superior performance compared with the state-of-the-art implementations on CPU, GPU, ACAP, and other custom GNN accelerators. Compared with these implementations, we achieve significant average runtime speedup across various models and datasets of 162.42x, 17.01x, 9.90x, and 27.23x, respectively. Furthermore, for Graph Convolutional Network (GCN) inference, our approach leads to a speedup of 3.9-96.7x compared to designs using PL only on the same ACAP device.