mlsanityData to 7 Oct 2026

Auditing Prompt Caching in Language Model APIs

arXiv 2502.07776 ↗First posted 11 Feb 2025cs.CL cs.CR cs.LGcited by 19

Not on the map. Why?

Abstract (arXiv)

Prompt caching in large language models (LLMs) results in data-dependent timing variations: cached prompts are processed faster than non-cached prompts. These timing differences introduce the risk of side-channel timing attacks. For example, if the cache is shared across users, an attacker could identify cached prompts from fast API response times to learn information about other users' prompts. Because prompt caching may cause privacy leakage, transparency around the caching policies of API providers is important. To this end, we develop and conduct statistical audits to detect prompt caching in real-world LLM API providers. We detect global cache sharing across users in seven API providers, including OpenAI, resulting in potential privacy leakage about users' prompts. Timing variations due to prompt caching can also result in leakage of information about model architecture. Namely, we find evidence that OpenAI's embedding model is a decoder-only Transformer, which was previously not publicly known.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 21 of its 48 references that are in the map. First posted on arXiv 11 Feb 2025.

Builds on

21 of 48 references.

References · 21

  1. 2017Attention Is All You Needcited by 59,932
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  4. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  5. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  6. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  7. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 975
  8. 2022MTEB: Massive Text Embedding Benchmarkcited by 623
  9. 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 255
  10. 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusioncited by 211
  11. 2024Mooncake: A KVCache-centric Disaggregated Architecture for LLM Servingcited by 193
  12. 2024Stealing Part of a Production Language Modelcited by 152
9 more
  1. 2024ChunkAttention: Efficient Self-Attention with Prefix-Aware KV Cache and Two-Phase Partitioncited by 81
  2. 2024Hydragen: High-Throughput LLM Inference with Shared Prefixescited by 71
  3. 2018Spectre Attacks: Exploiting Speculative Executioncited by 47
  4. 2024Logits of API-Protected LLMs Leak Proprietary Informationcited by 38
  5. 2024The Early Bird Catches the Leak: Unveiling Timing Side Channels in LLM Serving Systemscited by 29
  6. 2024What Was Your Prompt? A Remote Keylogging Attack on AI Assistantscited by 26
  7. 2024InputSnatch: Stealing Input in LLM Services via Timing Side-Channel Attackscited by 16
  8. 2024Remote Timing Attacks on Efficient Language Model Inferencecited by 13
  9. 2024When Speculation Spills Secrets: Side Channels via Speculative Decoding In LLMscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 14.

19 papers.

Sort:

By area:

  1. 2025A Study of LLMs' Preferences for Libraries and Programming Languagescited by 20
  2. 2026Don't Break the Cache: An Evaluation of Prompt Caching for Long-Horizon Agentic Taskscited by 18
  3. 2025Auditing Black-Box LLM APIs with a Rank-Based Uniformity Testcited by 16
  4. 2025Beyond Data Privacy: New Privacy Risks for Large Language Modelscited by 6
  5. 2026From Similarity to Vulnerability: Key Collision Attack on LLM Semantic Cachingcited by 3
  6. 2026TVCACHE: A Stateful Tool-Value Cache for Post-Training LLM Agentscited by 2
  7. 2025AttestLLM: Efficient Attestation Framework for Billion-scale On-device LLMscited by 1
  8. 2025Library Hallucinations in LLM-Generated Code: A Risk Analysis Grounded in Developer Queriescited by 1
  9. 2026OptiLeak: Efficient Prompt Reconstruction via Reinforcement Learning in Multi-tenant LLM Servicescited by 1
  10. 2026PrefixWall: Mitigating Prefix Caching Side Channels in Shared LLM Systemscited by 1
  11. 2026Behavioral Consistency and Transparency Analysis on Large Language Model API Gatewayscited by 1
  12. 2026CacheProbe: Auditing Prompt Cache Isolation in Gateway APIscited by 1
7 more
  1. 2026Cache-Aware Prompt Compression:A Two-Tier Cost Model for LLM API Cachingcited by 1
  2. 2026HijackKV: New Threat in Position-Independent KV Cache Reusecited by 1
  3. 2026LaCache: Robust Semantic Caching for LLM Servingcited by 1
  4. 2026Same Pieces, Different Servers: A Tetris Benchmark for AI Agents as Servedcited by 0
  5. 2026Shape and Substance: Dual-Layer Side-Channel Attacks on Local Vision-Language Modelscited by 0
  6. 2026The Periodic Table of LLM Reasoning: A Structured Survey of Reasoning Paradigms, Methods, and Failure Modescited by 0
  7. 2026Leaky Language Models: Stealing Architecture and Inference Optimizations via Per-Token Timingcited by 0