# Selective KV-Cache Sharing to Mitigate Timing Side-Channels in LLM Inference

arXiv: https://arxiv.org/abs/2508.08438 · HTML: https://mlsanity.com/paper/2508.08438/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving.
- Builds on 25 of its 57 references that are in the map.
- First posted on arXiv 11 Aug 2025.

## Placed in

- Efficient LLM inference and serving / [KV cache optimization](https://mlsanity.com/inference/kv-cache-optimization/)

## Builds on (most cited first, 25 of 25)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [SGLang: Efficient Execution of Structured Language Model Programs](https://mlsanity.com/paper/2312.07104/) (2023)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [A Survey of Transformers](https://mlsanity.com/paper/2106.04554/) (2021)
- [A Survey on Large Language Model Acceleration based on KV Cache Management](https://mlsanity.com/paper/2412.19442/) (2024)
- [ChunkAttention: Efficient Self-Attention with Prefix-Aware KV Cache and Two-Phase Partition](https://mlsanity.com/paper/2402.15220/) (2024)
- [Spectre Attacks: Exploiting Speculative Execution](https://mlsanity.com/paper/1801.01203/) (2018)
- [Stealing Neural Networks via Timing Side Channels](https://mlsanity.com/paper/1812.11720/) (2018)
- [Block Transformer: Global-to-Local Language Modeling for Fast Inference](https://mlsanity.com/paper/2406.02657/) (2024)
- [BatchLLM: Optimizing Large Batched LLM Inference with Global Prefix Sharing and Throughput-oriented Token Batching](https://mlsanity.com/paper/2412.03594/) (2024)
- [The Common Pile v0.1: An 8TB Dataset of Public Domain and Openly Licensed Text](https://mlsanity.com/paper/2506.05209/) (2025)
- [The Early Bird Catches the Leak: Unveiling Timing Side Channels in LLM Serving Systems](https://mlsanity.com/paper/2409.20002/) (2024)
- [InputSnatch: Stealing Input in LLM Services via Timing Side-Channel Attacks](https://mlsanity.com/paper/2411.18191/) (2024)
- [Configurable Safety Tuning of Language Models with Synthetic Preference Data](https://mlsanity.com/paper/2404.00495/) (2024)
- [Leaky Nets: Recovering Embedded Neural Network Models and Inputs through Simple Power and Timing Side-Channels -- Attacks and Defenses](https://mlsanity.com/paper/2103.14739/) (2021)
- [DePrompt: Desensitization and Evaluation of Personal Identifiable Information in Large Language Model Prompts](https://mlsanity.com/paper/2408.08930/) (2024)
- [BUZZ: Beehive-structured Sparse KV Cache with Segmented Heavy Hitters for Efficient LLM Inference](https://mlsanity.com/paper/2410.23079/) (2024)
- [Demystifying LLM Supply Chain Vulnerabilities in the Wild: Distribution, Root Cause, and Real-World Impact](https://mlsanity.com/paper/2502.12497/) (2025)
- [Time Will Tell: Timing Side Channels via Output Token Count in Large Language Models](https://mlsanity.com/paper/2412.15431/) (2024)
- [Enhancing the De-identification of Personally Identifiable Information in Educational Data](https://mlsanity.com/paper/2501.09765/) (2025)
- [A Desynchronization-Based Countermeasure Against Side-Channel Analysis of Neural Networks](https://mlsanity.com/paper/2303.18132/) (2023)
- [MPCache: MPC-Friendly KV Cache Eviction for Efficient Private LLM Inference](https://mlsanity.com/paper/2501.06807/) (2025)
- [A Placement Vulnerability Study in Multi-tenant Public Clouds](https://mlsanity.com/paper/1507.03114/) (2015)
- [Swivel: Hardening WebAssembly against Spectre](https://mlsanity.com/paper/2102.12730/) (2021)

## Built on by (most cited first, 18 of 18)

- [JanusVLN: Decoupling Semantics and Spatiality with Dual Implicit Memory for Vision-Language Navigation](https://mlsanity.com/paper/2509.22548/) (2025)
- [HUD: Hierarchical Uncertainty-Aware Disambiguation Network for Composed Video Retrieval](https://mlsanity.com/paper/2512.02792/) (2025)
- [SCRIBE: Structured Mid-Level Supervision for Tool-Using Language Models](https://mlsanity.com/paper/2601.03555/) (2026)
- [PromptSculptor: Multi-Agent Based Text-to-Image Prompt Optimization](https://mlsanity.com/paper/2509.12446/) (2025)
- [Learning How to Use Tools, Not Just When: Pattern-Aware Tool-Integrated Reasoning](https://mlsanity.com/paper/2509.23292/) (2025)
- [Can Transformer Memory Be Corrupted? Investigating Cache-Side Vulnerabilities in Large Language Models](https://mlsanity.com/paper/2510.17098/) (2025)
- [Whose Narrative is it Anyway? A KV Cache Manipulation Attack](https://mlsanity.com/paper/2511.12752/) (2025)
- [MarginGate: Sparse Margin-Triggered Verification for Batch-Invariant LLM Inference](https://mlsanity.com/paper/2605.30218/) (2026)
- [LLM Benchmark Datasets Should Be Contamination-Resistant](https://mlsanity.com/paper/2605.19999/) (2026)
- [Agent-Assisted Side-Channel Attacks on Non-Prefix KV Cache in RAG](https://mlsanity.com/paper/2606.21842/) (2026)
- [Coarse-to-Fine Personalized LLM Impressions for Streamlined Radiology Reports](https://mlsanity.com/paper/2508.15845/) (2025)
- [OptiLeak: Efficient Prompt Reconstruction via Reinforcement Learning in Multi-tenant LLM Services](https://mlsanity.com/paper/2602.20595/) (2026)
- [PrefixWall: Mitigating Prefix Caching Side Channels in Shared LLM Systems](https://mlsanity.com/paper/2603.10726/) (2026)
- [GaitProtector: Impersonation-Driven Gait De-Identification via Training-Free Diffusion Latent Optimization](https://mlsanity.com/paper/2605.12431/) (2026)
- [CoWeaver: A Bi-directional, Learnable and Explainable Matching Engine for Mixed Human-Agent Science Collaboration](https://mlsanity.com/paper/2607.15545/) (2026)
- [Characterizing Contention-Induced Reliability Collapse in KV-Cache Timing Side Channels for Multi-Tenant LLM Serving](https://mlsanity.com/paper/2609.06853/) (2026)
- [When the Agent Becomes the Kernel: A Systematization of Security on the Path to AI-Native Operating Systems](https://mlsanity.com/paper/2609.23700/) (2026)
- [SparLeak: Privacy Leakage from Sparse Attention in LLM Inference on Shared GPUs](https://mlsanity.com/paper/2609.38830/) (2026)

## Abstract (arXiv)

Global KV-cache sharing is an effective optimization for accelerating large language model (LLM) inference, yet it introduces an API-visible timing side channel that lets adversaries infer sensitive user inputs from shared entries, leading to cross-tenant privacy risks. To address this problem, we introduce SafeKV (Secure and Flexible KV-cache Sharing), a system-level co-design of privacy enforcement and KV-cache management. SafeKV integrates lightweight detection and isolation directly into the serving runtime to eliminate cross-tenant reuse of sensitive KV-cache blocks under our threat model, while recovering most of the performance benefits of global sharing. Our key contributions are: (1) a three-tier asynchronous detection pipeline that decouples privacy classification from inference and supports streaming workloads, (2) a unified radix-tree-based memory manager with path compression and sensitivity-aware eviction for scalable selective isolation, and (3) an RDR-guided (Reuse Diversity Ratio) runtime safeguard that detects and bounds residual leakage. On large LLM backends, SafeKV reduces the time-to-first-token (TTFT) overhead compared to full isolation by up to 40.58% and raises throughput by up to 2.66x. Overall, SafeKV restores the efficiency of KV reuse while enforcing strong, practical privacy for multi-tenant LLM inference.