mlsanityData to 14 Oct 2026

HBFlex: A Flexible Memory System for Bridging Fine-Grained LLM States and Coarse-Grained HBF Parallel Execution

arXiv 2609.18675 ↗First posted 16 Sep 2026cs.ARcited by 1

On the map: Efficient LLM inference and serving → Serving systems and request scheduling

Abstract (arXiv)

Large language models (LLMs) require increasing memory capacity to accommodate growing model weights and KV caches. High-Bandwidth Flash (HBF) offers high memory density and aggregate read bandwidth through massive plane-level parallelism, making it an attractive option for LLM serving. However, serving LLMs entirely from HBF introduces three challenges: fine-grained KV reads create placement and access imbalance, incremental writes interfere with foreground reads, and mixed KV lifetimes amplify garbage collection. Hybrid HBM/HBF designs retain HBM to support dynamic KV management, but this allocation reduces the HBF resources available under a fixed packaging budget, limiting aggregate HBF bandwidth. We present HBFlex, a full-HBF memory system with coordinated optimizations for KV reads, writes, and reclamation. HBFlex balances KV placement and attention accesses to improve plane utilization. It aggregates incremental updates and schedules writeback within sufficiently long compute windows to reduce write--read interference. It also combines lifetime-guided block packing with deferred reclamation to reduce valid-page migration. We evaluate HBFlex through trace-driven simulation across different configurations. HBFlex achieves average throughput speedups of up to 1.58× over FlashAccel and 3.30× over H3, benefiting from higher HBF bandwidth and more efficient management of dynamic KV-cache reads, writes, and erases.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 23 of its 42 references that are in the map. First posted on arXiv 16 Sep 2026.

Builds on

23 of 42 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026Potential Applications of HBF in LLM Serving Systems13 Aug
  2. 2026FlashAccel: Leveraging High-Bandwidth Flash (HBF) for High-Throughput LLM Inference11 Jul
  3. 2024Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving24 Jun
  4. 2024vAttention: Dynamic Memory Management for Serving LLMs without PagedAttention7 May
  5. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve4 Mar
  6. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving18 Jan

Other references · 17

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  4. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  5. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  6. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  7. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  8. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  9. 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
  10. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 558
  11. 2022DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scalecited by 407
  12. 2025FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Servingcited by 258
5 more
  1. 2024A Survey on Recent Advances in LLM-Based Multi-turn Dialogue Systemscited by 170
  2. 2024Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencentcited by 90
  3. 2025KVCache Cache in the Wild: Characterizing and Optimizing KVCache Cache at a Large Cloud Providercited by 41
  4. 2026Challenges and Research Directions for Large Language Model Inference Hardwarecited by 11
  5. 2026HBF Sucks? A Full-Stack Characterization of High-Bandwidth Flash for KV-Centric LLM Servingcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026SPLASH: Co-Designing Sparse Attention with High-Bandwidth Flash for Efficient Long-Context Inferencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Characterizing High Bandwidth Flash for LLM Serving
  2. 2026Hot-Cold Tiering of HBM and High Bandwidth Flash for Agentic LLM Serving
  3. 2026FLINT: Efficiently Leveraging High Bandwidth Flash for Capacity-Scalable LLM Inference Acceleration
  4. 2025Accelerating LLM Inference via Dynamic KV Cache Placement in Heterogeneous Memory System
  5. 2026TokenStack: A Heterogeneous HBM-PIM Architecture and Runtime for Efficient LLM Inference
  6. 2026Adaptive Multi-Objective Tiered Storage Configuration for KV Cache in LLM Service
  7. 2026A CXL Memory Rack for Multi-Turn LLM Serving
  8. 2026Beyond Capacity: Scalable MoE LLM Inference via High-Bandwidth Flash with Direct GPU and HBM Paths