mlsanityData to 14 Oct 2026

Potential Applications of HBF in LLM Serving Systems

arXiv 2608.13127 ↗First posted 13 Aug 2026cs.ARcited by 2

On the map: Efficient LLM inference and serving → Serving systems and request scheduling

Abstract (arXiv)

LLM serving is increasingly constrained by memory capacity as model weights, KV caches, and the number of served model variants continue to grow. This report examines High-Bandwidth Flash (HBF) as a capacity-oriented extension to HBM-based serving systems. We first discuss how HBF can be integrated into the GPU memory hierarchy without undermining the bandwidth expected by the compute die. We then model the system-level value of added capacity as expanded residency for read-mostly model-state objects. Under this view, HBF can improve MoE serving by enabling more expert replicas and can improve multi-model serving by reducing model loading and supporting hot-model replication. Our simulation results show that these benefits depend on preserving the HBM-resident execution path while using HBF to expand the resident set of model weights.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 8 of its 16 references that are in the map. First posted on arXiv 13 Aug 2026.

Builds on

8 of 16 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024MuxServe: Flexible Spatial-Temporal Multiplexing for Multiple LLM Serving2 Apr
  2. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve4 Mar
  3. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving18 Jan

Other references · 5

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  2. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 575
  3. 2024ServerlessLLM: Low-Latency Serverless Inference for Large Language Modelscited by 80
  4. 2026Challenges and Research Directions for Large Language Model Inference Hardwarecited by 11
  5. 2025Efficient MoE Serving in the Memory-Bound Regime: Balance Activated Experts, Not Tokenscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026HBFlex: A Flexible Memory System for Bridging Fine-Grained LLM States and Coarse-Grained HBF Parallel Executioncited by 1
  2. 2026SPLASH: Co-Designing Sparse Attention with High-Bandwidth Flash for Efficient Long-Context Inferencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Characterizing High Bandwidth Flash for LLM Serving
  2. 2026HBFSim: Fast and Faithful Simulation of High-Bandwidth Flash Under Real GPU Execution ↗
  3. 2026Beyond Capacity: Scalable MoE LLM Inference via High-Bandwidth Flash with Direct GPU and HBM Paths
  4. 2026HBF Sucks? A Full-Stack Characterization of High-Bandwidth Flash for KV-Centric LLM Serving
  5. 2026FlashAccel: Leveraging High-Bandwidth Flash (HBF) for High-Throughput LLM Inference
  6. 2026FLINT: Efficiently Leveraging High Bandwidth Flash for Capacity-Scalable LLM Inference Acceleration
  7. 2026Hot-Cold Tiering of HBM and High Bandwidth Flash for Agentic LLM Serving
  8. 2024KunServe: Parameter-centric Memory Management for Efficient Memory Overloading Handling in LLM Serving