mlsanityData to 14 Oct 2026

SAC: Disaggregated KV Cache System for Sparse Attention LLMs with CXL

arXiv 2606.19746 ↗First posted 18 Jun 2026cs.DCcited by 2

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

The scaling of LLMs toward long-context inference has shifted the primary serving system bottleneck from computation to memory capacity. Traditional solutions for dense attention models rely on RDMA-based disaggregated memory pools, which perform coarse-grained fetching of the entire prefix KV cache from remote storage to local memory before decoding. However, this approach is fundamentally inefficient for emerging sparse attention models. While only a small fraction of KV entries are active during decoding, these systems still fetch the full KV cache locally, leading to severe transmission bottlenecks and local memory wastage. To address this, we propose SAC, the first efficient disaggregated KV cache system optimized for sparse attention models. By leveraging the low-latency, cache-line granularity load/store semantics of Compute Express Link (CXL), SAC fetches only the required top-k KV entries on demand during inference. Evaluations on DeepSeek-V3.2 using SGLang show that SAC achieves 2.1x higher throughput, 9.7x lower TTFT, and 1.8x lower TBT compared to RDMA-based baselines, establishing CXL-based disaggregation as the superior infrastructure for emerging sparse attention models.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 17 of its 33 references that are in the map. First posted on arXiv 18 Jun 2026.

Builds on

17 of 33 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025TraCT: Disaggregated LLM Serving with CXL Shared Memory KV Cache at Rack-Scale20 Dec
  2. 2025Beluga: A CXL-Based Memory Architecture for Scalable and Efficient LLM KVCache Management25 Nov
  3. 2024ClusterKV: Manipulating LLM KV Cache in Semantic Space for Recallable Compression4 Dec
  4. 2024BUZZ: Beehive-structured Sparse KV Cache with Segmented Heavy Hitters for Efficient LLM Inference30 Oct
  5. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference16 Jun
  6. 2024SnapKV: LLM Knows What You are Looking for Before Generation22 Apr
  7. 2023CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving11 Oct
  8. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models24 Jun

Other references · 9

  1. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  2. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  3. 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 815
  4. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 617
  5. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 575
  6. 2024A Survey on Efficient Inference for Large Language Modelscited by 205
  7. 2023An Introduction to the Compute Express Link (CXL) Interconnectcited by 9
  8. 2024KVDirect: Distributed Disaggregated LLM Inferencecited by 8
  9. 2025UB-Mesh: a Hierarchically Localized nD-FullMesh Datacenter Network Architecturecited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Heterogeneous LLM Serving with General-Purpose Processing-Near-Memory for Retrieval-Based Sparse Attentioncited by 0
  2. 2026Composable CXL Memory as a Kubernetes-Native Shared Memory for LLM Servingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025SmallKV: Small Model Assisted Compensation of KV Cache Compression for Efficient LLM Inference
  2. 2025HCAttention: Extreme KV Cache Compression via Heterogeneous Attention Computing for LLMs
  3. 2026OasisKV: Scaling In-Decode KV Cache Beyond HBM with Lookahead Sparse Prefetching
  4. 2026Unifying Sparse Attention with Hierarchical Memory for Scalable Long-Context LLM Serving
  5. 2026Sparse Attention as a Range Searching Problem: Towards an Inference-Efficient Index for KV Cache
  6. 2025Progressive Sparse Attention: Algorithm and System Co-design for Efficient Attention in LLM Serving
  7. 2025NOSA: Native and Offloadable Sparse Attention
  8. 2025KVSink: Understanding and Enhancing the Preservation of Attention Sinks in KV Cache Quantization for LLMs