# Exploring High-Bandwidth Flash for Modern LLM Inference: Opportunities and Challenges

arXiv: https://arxiv.org/abs/2608.13868 · HTML: https://mlsanity.com/paper/2608.13868/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Aug 2026.

## Built on by (most cited first, 4 of 4)

- [FLINT: Efficiently Leveraging High Bandwidth Flash for Capacity-Scalable LLM Inference Acceleration](https://mlsanity.com/paper/2608.25062/) (2026)
- [Enabling High-Bandwidth Flash for Generative Recommendation Serving with Write-Aware KV Cache Policy](https://mlsanity.com/paper/2609.07175/) (2026)
- [SPLASH: Co-Designing Sparse Attention with High-Bandwidth Flash for Efficient Long-Context Inference](https://mlsanity.com/paper/2609.23816/) (2026)
- [Hot-Cold Tiering of HBM and High Bandwidth Flash for Agentic LLM Serving](https://mlsanity.com/paper/2609.25782/) (2026)

## Abstract (arXiv)

This work investigates the potential benefits and technical challenges of using high-bandwidth flash (HBF) for large language model (LLM) inference. HBF has gained increasing attention as a promising solution to mitigate memory-capacity bottlenecks in modern LLM-serving systems, but its benefits and challenges remain largely uninvestigated. To address this gap, we thoroughly analyze HBF-based LLM-serving systems under diverse system configurations and operating scenarios in which HBF serves as a main GPU-memory component to handle both reads and writes. Our analysis shows that, despite its limited write performance, HBF can significantly improve the batch size, throughput, and flexibility of LLM-serving systems while reducing the minimum GPU requirements, but realizing these benefits critically depends on sustaining HBM-comparable read bandwidth and requires significant endurance improvements.