# DCI: A Coordinated Allocation and Filling Workload-Aware Dual-Cache Allocation GNN Inference Acceleration System

arXiv: https://arxiv.org/abs/2503.01281 · HTML: https://mlsanity.com/paper/2503.01281/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving.
- Builds on 15 of its 33 references that are in the map.
- First posted on arXiv 3 Mar 2025.

## Placed in

- Efficient LLM inference and serving / [KV cache optimization](https://mlsanity.com/inference/kv-cache-optimization/)

## Builds on (most cited first, 15 of 15)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling](https://mlsanity.com/paper/1801.10247/) (2018)
- [Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks](https://mlsanity.com/paper/1905.07953/) (2019)
- [GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training](https://mlsanity.com/paper/2006.09963/) (2020)
- [Graph Neural Network for Traffic Forecasting: A Survey](https://mlsanity.com/paper/2101.11174/) (2021)
- [Adaptive Sampling Towards Fast Graph Representation Learning](https://mlsanity.com/paper/1809.05343/) (2018)
- [Large Graph Convolutional Network Training with GPU-Oriented Data Communication Architecture](https://mlsanity.com/paper/2103.03330/) (2021)
- [Efficient Graph Neural Network Inference at Large Scale](https://mlsanity.com/paper/2211.00495/) (2022)
- [GNNSampler: Bridging the Gap between Sampling Algorithms of GNN and Hardware](https://mlsanity.com/paper/2108.11571/) (2021)
- [NeutronOrch: Rethinking Sample-based GNN Training under CPU-GPU Heterogeneous Environments](https://mlsanity.com/paper/2311.13225/) (2023)
- [GLISP: A Scalable GNN Learning System by Exploiting Inherent Structural Properties of Graphs](https://mlsanity.com/paper/2401.03114/) (2024)

## Abstract (arXiv)

Graph Neural Networks (GNNs) are powerful tools for processing graph-structured data, increasingly used for large-scale real-world graphs via sampling-based inference methods. However, inherent characteristics of neighbor sampling lead to redundant data loading during GNN inference, compounded by inefficient data transfers between host and GPU memory, resulting in slow inference and low resource utilization. Existing methods to accelerate GNN inference face several challenges: (1) low practical GPU memory utilization, (2) overlooking adjacency matrix locality, and (3) long preprocessing time. To address these challenges, we introduce DCI, an efficient workload-aware dual-cache allocation system for GNN inference acceleration. DCI allocates cache capacities for both node features and adjacency matrices based on workload patterns during the pre-sampling phase, leveraging a lightweight cache-filling algorithm to optimize data loading efficiency. Experimental results demonstrate that DCI accelerates sampling and node feature loading, achieving end-to-end inference speedups of 1.18× to 11.26× compared to DGL, and 1.14× to 13.68× over RAIN, while reducing preprocessing time by 52.8% to 98.7%. Additionally, DCI outperforms state-of-the-art single-cache inference systems by achieving speedup of 1.08× to 1.32×. We also compared DCI with DUCATI's dual-cache population strategy. Our lightweight population algorithm allows DCI to achieve nearly the same inference speed while keeping preprocessing time to less than 20% of that required by DUCATI.