# GPU-Initiated On-Demand High-Throughput Storage Access in the BaM System Architecture

arXiv: https://arxiv.org/abs/2203.04910 · HTML: https://mlsanity.com/paper/2203.04910/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Mar 2022.

## Built on by (most cited first, 12 of 12)

- [Strata: Hierarchical Context Caching for Long Context Language Model Serving](https://mlsanity.com/paper/2508.18572/) (2025)
- [Smart-Infinity: Fast Large Language Model Training using Near-Storage Processing on a Real System](https://mlsanity.com/paper/2403.06664/) (2024)
- [Beluga: A CXL-Based Memory Architecture for Scalable and Efficient LLM KVCache Management](https://mlsanity.com/paper/2511.20172/) (2025)
- [LithOS: An Operating System for Efficient Machine Learning on GPUs](https://mlsanity.com/paper/2504.15465/) (2025)
- [Accelerating Sampling and Aggregation Operations in GNN Frameworks with GPU Initiated Direct Storage Accesses](https://mlsanity.com/paper/2306.16384/) (2023)
- [A Cost-Effective Near-Storage Processing Solution for Offline Inference of Long-Context LLMs](https://mlsanity.com/paper/2502.09921/) (2025)
- [Cost-Efficient LLM Training with Lifetime-Aware Tensor Offloading via GPUDirect Storage](https://mlsanity.com/paper/2506.06472/) (2025)
- [DUAL-BLADE: Dual-Path NVMe-Direct KV-Cache Offloading for Edge LLM Inference](https://mlsanity.com/paper/2604.26557/) (2026)
- [Reducing Memory Contention and I/O Congestion for Disk-based GNN Training](https://mlsanity.com/paper/2406.13984/) (2024)
- [LSM-GNN: Large-scale Storage-based Multi-GPU GNN Training by Optimizing Data Transfer Scheme](https://mlsanity.com/paper/2407.15264/) (2024)
- [Reducing the GPU Memory Bottleneck with Lossless Compression for ML -- Extended](https://mlsanity.com/paper/2605.30728/) (2026)
- [Building py-kvcache: A Performance Characterization of External KV Caching for vLLM with NVMe SSDs](https://mlsanity.com/paper/2609.11744/) (2026)

## Abstract (arXiv)

Graphics Processing Units (GPUs) have traditionally relied on the host CPU to initiate access to the data storage. This approach is well-suited for GPU applications with known data access patterns that enable partitioning of their dataset to be processed in a pipelined fashion in the GPU. However, emerging applications such as graph and data analytics, recommender systems, or graph neural networks, require fine-grained, data-dependent access to storage. CPU initiation of storage access is unsuitable for these applications due to high CPU-GPU synchronization overheads, I/O traffic amplification, and long CPU processing latencies. GPU-initiated storage removes these overheads from the storage control path and, thus, can potentially support these applications at much higher speed. However, there is a lack of systems architecture and software stack that enable efficient GPU-initiated storage access. This work presents a novel system architecture, BaM, that fills this gap. BaM features a fine-grained software cache to coalesce data storage requests while minimizing I/O traffic amplification. This software cache communicates with the storage system via high-throughput queues that enable the massive number of concurrent threads in modern GPUs to make I/O requests at a high rate to fully utilize the storage devices and the system interconnect. Experimental results show that BaM delivers 1.0x and 1.49x end-to-end speed up for BFS and CC graph analytics benchmarks while reducing hardware costs by up to 21.7x over accessing the graph data from the host memory. Furthermore, BaM speeds up data-analytics workloads by 5.3x over CPU-initiated storage access on the same hardware.