mlsanityData to 14 Oct 2026

Composable CXL Memory as a Kubernetes-Native Shared Memory for LLM Serving

arXiv 2609.10790 ↗First posted 9 Sep 2026cs.DC cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We present a Kubernetes Dynamic Resource Allocation (DRA) driver that makes composable CXL memory a schedulable cluster resource, and evaluate the resulting shared-memory tier for cross-node KV-cache reuse in LLM serving. The driver composes CXL regions on demand, materializes them as DAX devices on each participating host, and injects them into pods under a single Container Device Interface (CDI) name so that pods on different nodes access the same physical region. A shared-memory connector for vLLM/llm-d uses that region as a KV-cache tier with a slot directory embedded inside the shared medium, which eliminates the need for an external metadata service. On a two-node cluster with a 512,GiB CXL appliance and Qwen2.5-7B-Instruct, cross-node prefix reuse reduces TTFT by 5.5×--36.6× at an external hit rate of 95.4--99.5,%, while node-local tiers (GPU prefix caching, CPU-DRAM offload) fall back to full recompute. The sharing gap, defined as the latency ratio between cross-node and same-node reuse, is 1--4%, indicating that cross-node reuse incurs little additional latency relative to same-node reuse on our testbed. Both replicas run full engines; the study demonstrates memory disaggregation rather than prefill/decode disaggregation. We report this as a feasibility study rather than a performance evaluation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 14 of its 38 references that are in the map. First posted on arXiv 9 Sep 2026.

Builds on

14 of 38 references.

References · 14

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  2. 2024Qwen2.5 Technical Reportcited by 4,942
  3. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  4. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 617
  5. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 558
  6. 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusioncited by 223
  7. 2025LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inferencecited by 126
  8. 2025TraCT: Disaggregated LLM Serving with CXL Shared Memory KV Cache at Rack-Scalecited by 13
  9. 2022TPP: Transparent Page Placement for CXL-Enabled Tiered-Memorycited by 10
  10. 2025A Survey of LLM Inference Systemscited by 9
  11. 2023Demystifying CXL Memory with Genuine CXL-Ready Systems and Devicescited by 6
  12. 2025Octopus: Enhancing CXL Memory Pods via Sparse Topologycited by 3
2 more
  1. 2026SAC: Disaggregated KV Cache System for Sparse Attention LLMs with CXLcited by 2
  2. 2024The Hitchhiker's Guide to Programming and Optimizing Cache Coherent Heterogeneous Systems: CXL, NVLink-C2C, and AMD Infinity Fabriccited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026A CXL Memory Rack for Multi-Turn LLM Serving
  2. 2026Bridging LLM Serving and CXL-SSDs with Chunk-Aware KV Cache Management
  3. 2026ObjectCache: Layerwise Object-Storage Retrieval for KV Cache Reuse
  4. 2025Beluga: A CXL-Based Memory Architecture for Scalable and Efficient LLM KVCache Management
  5. 2026Elastic KV Cache for LLM Serving:A Working Reclamation Mechanism, and Why Chunked Prefill Already Closes the Gap
  6. 2026Building py-kvcache: A Performance Characterization of External KV Caching for vLLM with NVMe SSDs
  7. 2026DualMap: Enabling Both Cache Affinity and Load Balancing for Distributed LLM Serving
  8. 2025Scalable Processing-Near-Memory for 1M-Token LLM Inference: CXL-Enabled KV-Cache Management Beyond GPU Limits