# ESS: An Offload-Centric Latent-Cache Management Architecture for DeepSeek-V3.2-Exp

arXiv: https://arxiv.org/abs/2512.10576 · HTML: https://mlsanity.com/paper/2512.10576/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Dec 2025.

## Built on by (most cited first, 1 of 1)

- [HiSparse: Scaling Sparse-Attention Decoding with Hierarchical KV Cache Management](https://mlsanity.com/paper/2608.07009/) (2026)

## Abstract (arXiv)

DeepSeek-V3.2-Exp introduces a sparse attention mechanism that significantly reduces inference latency in long-context scenarios. Although the overall throughput has improved greatly, the Decode-stage of PD disaggregation remains to be a major bottleneck. This bottleneck primarily stems from the conflict between linear growth of Latent-Cache with sequence length and the limited GPU memory capacity, which constrains the feasible batch-size and thereby suppresses Decode-stage throughput. To address this challenge, we propose ESS (Extended Sparse Server), an offload-centric system design tailored for DeepSeek-V3.2-Exp. ESS selectively offloads Latent-Cache to CPU memory while preserving latency-critical components on GPU. By freeing up GPU memory, ESS effectively decoupling batch-size scaling from GPU memory constraints. This design significantly improves Decode-stage throughput, thereby reducing deployment costs in real-world settings. Our high-fidelity simulations show that ESS delivers 69.4% throughput improvement at 32K context length and up to 123% throughput improvement at 128K, demonstrating its effectiveness for large-context inference workloads. These results highlight ESS as a practical and scalable solution for long-context LLM serving.