PrefixWall: Mitigating Prefix Caching Side Channels in Shared LLM Systems
On the map: Efficient LLM inference and serving → KV cache optimization
Abstract (arXiv)
Large Language Models (LLMs) rely on optimizations like Automatic Prefix Caching (APC) to accelerate inference. APC works by reusing previously computed states for the beginning part of a request (prefix), when another request starts with the same text. While APC improves throughput, it introduces timing side channels: cache hits are faster than misses, creating observable latency differences. In multi-tenant systems, attackers can exploit these differences to infer sensitive information, e.g., by incrementally reconstructing another user's request by observing hit/miss patterns. Current defenses take a sledgehammer approach: they disable APC and cache sharing, isolating users, and sacrificing efficiency for regular users. This paper presents PrefixWall, a system that secures multi-tenant LLM serving systems against APC side channels without sacrificing performance and efficiency. PrefixWall monitors cache reuse across users, flags suspicious sharing, and selectively isolates prefixes, restricting their reuse only when necessary. Evaluation shows that PrefixWall enables up to 70% higher cache reuse and 30% lower inference latency compared to existing defenses that isolate users. PrefixWall's lightweight design demonstrates how security in LLM serving does not have to come at the cost of unnecessarily reduced performance or unbearable overheads.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 34 of its 71 references that are in the map. First posted on arXiv 11 Mar 2026.
Builds on
34 of 71 references.
Closest ancestors · 8
Same direction, newest first
- 2025Selective KV-Cache Sharing to Mitigate Timing Side-Channels in LLM Inference11 Aug
- 2024Discovering the Gems in Early Layers: Accelerating Long-Context LLMs with 1000x Input Token Reduction25 Sep
- 2024InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management28 Jun
- 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion26 May
- 2024Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttention23 Mar
- 2023Stateful Large Language Model Serving with Pensieve9 Dec
- 2023CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving11 Oct
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttention12 Sep
Other references · 26
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
- 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
- 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 595
- 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 536
- 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Servecited by 507
- 2023Ring Attention with Blockwise Transformers for Near-Infinite Contextcited by 448
- 2024MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attentioncited by 375
- 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 255
- 2023SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefillscited by 187
14 more
- 2024DeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inferencecited by 106
- 2024Hydragen: High-Throughput LLM Inference with Shared Prefixescited by 71
- 2024LazyLLM: Dynamic Token Pruning for Efficient Long Context LLM Inferencecited by 70
- 2024SampleAttention: Near-Lossless Acceleration of Long Context LLM Inference with Adaptive Structured Sparse Attentioncited by 39
- 2024The Early Bird Catches the Leak: Unveiling Timing Side Channels in LLM Serving Systemscited by 29
- 2024HashAttention: Semantic Sparsity for Faster Inferencecited by 23
- 2025Auditing Prompt Caching in Language Model APIscited by 19
- 2024Remote Timing Attacks on Efficient Language Model Inferencecited by 13
- 2024Configurable Safety Tuning of Language Models with Synthetic Preference Datacited by 12
- 2015DRAMA: Exploiting DRAM Addressing for Cross-CPU Attackscited by 10
- 2021Lord of the Ring(s): Side Channel Attacks on the CPU On-Chip Ring Interconnect Are Practicalcited by 3
- 2019Page Cache Attackscited by 2
- 2019Pacer: Comprehensive Network Side-Channel Mitigation in the Cloudcited by 2
- 2021Pool-Party: Exploiting Browser Resource Pools as Side-Channels for Web Trackingcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Sparse Prefix Caching for Hybrid and Recurrent LLM Serving
- 2026Characterizing Contention-Induced Reliability Collapse in KV-Cache Timing Side Channels for Multi-Tenant LLM Serving
- 2026When Fancy Eviction Fails: Rethinking Cache Replacement For LLM Prefix Reuse
- 2026Not All Tokens Are Worth Caching: Learning Semantic-Aware Eviction for LLM Prefix Caches
- 2025TokenLake: A Unified Segment-level Prefix Cache Pool for Fine-grained Elastic Long-Context LLM Serving
- 2026PrefixBench-H100: Characterizing Prefix Reuse and Time-to-First-Token in H100 LLM Serving
- 2026CacheRoute: Planned Prefix-Affinity Routing for Large-Scale LLM Serving
- 2026Governing the KV Cache: Preventing Timing Side-Channel Leakage in Multi-Tenant LLM Inference