mlsanityData to 14 Oct 2026

Selective KV-Cache Sharing to Mitigate Timing Side-Channels in LLM Inference

arXiv 2508.08438 ↗First posted 11 Aug 2025cs.CR cs.LG cs.OScited by 18

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

Global KV-cache sharing is an effective optimization for accelerating large language model (LLM) inference, yet it introduces an API-visible timing side channel that lets adversaries infer sensitive user inputs from shared entries, leading to cross-tenant privacy risks. To address this problem, we introduce SafeKV (Secure and Flexible KV-cache Sharing), a system-level co-design of privacy enforcement and KV-cache management. SafeKV integrates lightweight detection and isolation directly into the serving runtime to eliminate cross-tenant reuse of sensitive KV-cache blocks under our threat model, while recovering most of the performance benefits of global sharing. Our key contributions are: (1) a three-tier asynchronous detection pipeline that decouples privacy classification from inference and supports streaming workloads, (2) a unified radix-tree-based memory manager with path compression and sensitivity-aware eviction for scalable selective isolation, and (3) an RDR-guided (Reuse Diversity Ratio) runtime safeguard that detects and bounds residual leakage. On large LLM backends, SafeKV reduces the time-to-first-token (TTFT) overhead compared to full isolation by up to 40.58% and raises throughput by up to 2.66x. Overall, SafeKV restores the efficiency of KV reuse while enforcing strong, practical privacy for multi-tenant LLM inference.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 25 of its 57 references that are in the map. First posted on arXiv 11 Aug 2025.

Builds on

25 of 57 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024A Survey on Large Language Model Acceleration based on KV Cache Management27 Dec
  2. 2024BUZZ: Beehive-structured Sparse KV Cache with Segmented Heavy Hitters for Efficient LLM Inference30 Oct
  3. 2024Block Transformer: Global-to-Local Language Modeling for Fast Inference4 Jun
  4. 2024ChunkAttention: Efficient Self-Attention with Prefix-Aware KV Cache and Two-Phase Partition23 Feb
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttention12 Sep

Other references · 20

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  2. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  3. 2021Transformers in Vision: A Surveycited by 1,013
  4. 2021A Survey of Transformerscited by 401
  5. 2018Spectre Attacks: Exploiting Speculative Executioncited by 47
  6. 2018Stealing Neural Networks via Timing Side Channelscited by 43
  7. 2024BatchLLM: Optimizing Large Batched LLM Inference with Global Prefix Sharing and Throughput-oriented Token Batchingcited by 32
  8. 2025The Common Pile v0.1: An 8TB Dataset of Public Domain and Openly Licensed Textcited by 31
  9. 2024The Early Bird Catches the Leak: Unveiling Timing Side Channels in LLM Serving Systemscited by 29
  10. 2024InputSnatch: Stealing Input in LLM Services via Timing Side-Channel Attackscited by 16
  11. 2024Configurable Safety Tuning of Language Models with Synthetic Preference Datacited by 14
  12. 2021Leaky Nets: Recovering Embedded Neural Network Models and Inputs through Simple Power and Timing Side-Channels -- Attacks and Defensescited by 11
8 more
  1. 2024DePrompt: Desensitization and Evaluation of Personal Identifiable Information in Large Language Model Promptscited by 9
  2. 2025Demystifying LLM Supply Chain Vulnerabilities in the Wild: Distribution, Root Cause, and Real-World Impactcited by 8
  3. 2024Time Will Tell: Timing Side Channels via Output Token Count in Large Language Modelscited by 7
  4. 2025Enhancing the De-identification of Personally Identifiable Information in Educational Datacited by 7
  5. 2023A Desynchronization-Based Countermeasure Against Side-Channel Analysis of Neural Networkscited by 5
  6. 2025MPCache: MPC-Friendly KV Cache Eviction for Efficient Private LLM Inferencecited by 3
  7. 2015A Placement Vulnerability Study in Multi-tenant Public Cloudscited by 2
  8. 2021Swivel: Hardening WebAssembly against Spectrecited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 11.

18 papers.

Sort:

By area:

  1. 2025JanusVLN: Decoupling Semantics and Spatiality with Dual Implicit Memory for Vision-Language Navigationcited by 89
  2. 2025HUD: Hierarchical Uncertainty-Aware Disambiguation Network for Composed Video Retrievalcited by 32
  3. 2026SCRIBE: Structured Mid-Level Supervision for Tool-Using Language Modelscited by 21
  4. 2025PromptSculptor: Multi-Agent Based Text-to-Image Prompt Optimizationcited by 19
  5. 2025Learning How to Use Tools, Not Just When: Pattern-Aware Tool-Integrated Reasoningcited by 15
  6. 2025Can Transformer Memory Be Corrupted? Investigating Cache-Side Vulnerabilities in Large Language Modelscited by 3
  7. 2025Whose Narrative is it Anyway? A KV Cache Manipulation Attackcited by 3
  8. 2026MarginGate: Sparse Margin-Triggered Verification for Batch-Invariant LLM Inferencecited by 3
  9. 2026LLM Benchmark Datasets Should Be Contamination-Resistantcited by 2
  10. 2026Agent-Assisted Side-Channel Attacks on Non-Prefix KV Cache in RAGcited by 2
  11. 2025Coarse-to-Fine Personalized LLM Impressions for Streamlined Radiology Reportscited by 1
  12. 2026OptiLeak: Efficient Prompt Reconstruction via Reinforcement Learning in Multi-tenant LLM Servicescited by 1
6 more
  1. 2026PrefixWall: Mitigating Prefix Caching Side Channels in Shared LLM Systemscited by 1
  2. 2026GaitProtector: Impersonation-Driven Gait De-Identification via Training-Free Diffusion Latent Optimizationcited by 1
  3. 2026CoWeaver: A Bi-directional, Learnable and Explainable Matching Engine for Mixed Human-Agent Science Collaborationcited by 0
  4. 2026Characterizing Contention-Induced Reliability Collapse in KV-Cache Timing Side Channels for Multi-Tenant LLM Servingcited by 0
  5. 2026When the Agent Becomes the Kernel: A Systematization of Security on the Path to AI-Native Operating Systemscited by 0
  6. 2026SparLeak: Privacy Leakage from Sparse Attention in LLM Inference on Shared GPUscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Shadow in the Cache: Unveiling and Mitigating Privacy Risks of KV-cache in LLM Inference
  2. 2026Governing the KV Cache: Preventing Timing Side-Channel Leakage in Multi-Tenant LLM Inference
  3. 2025LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference
  4. 2026SmartGen: Seamless Disaggregated LLM Inference with Selective KV Cache Transfer
  5. 2024A Systematic Study of Cross-Layer KV Sharing for Efficient LLM Inference
  6. 2026KV Cache Optimization Strategies for Scalable and Efficient LLM Inference
  7. 2025FreeKV: Boosting KV Cache Retrieval for Efficient LLM Inference
  8. 2026HillInfer: Efficient Long-Context LLM Inference on the Edge with Hierarchical KV Eviction using SmartSSD