# Remote Timing Attacks on Efficient Language Model Inference

arXiv: https://arxiv.org/abs/2410.17175 · HTML: https://mlsanity.com/paper/2410.17175/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving.
- Builds on 20 of its 54 references that are in the map.
- First posted on arXiv 22 Oct 2024.

## Placed in

- Efficient LLM inference and serving / [Speculative decoding](https://mlsanity.com/inference/speculative-decoding/)

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads](https://mlsanity.com/paper/2401.10774/) (2024)
- [SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification](https://mlsanity.com/paper/2305.09781/) (2023)
- [Blockwise Parallel Decoding for Deep Autoregressive Models](https://mlsanity.com/paper/1811.03115/) (2018)
- [Break the Sequential Dependency of LLM Inference Using Lookahead Decoding](https://mlsanity.com/paper/2402.02057/) (2024)
- [Mixture-of-Depths: Dynamically allocating compute in transformer-based language models](https://mlsanity.com/paper/2404.02258/) (2024)
- [Accelerating Transformer Inference for Translation via Parallel Decoding](https://mlsanity.com/paper/2305.10427/) (2023)
- [CLLMs: Consistency Large Language Models](https://mlsanity.com/paper/2403.00835/) (2024)
- [Deep Fingerprinting: Undermining Website Fingerprinting Defenses with Deep Learning](https://mlsanity.com/paper/1801.02265/) (2018)
- [Speculative Streaming: Fast LLM Inference without Auxiliary Models](https://mlsanity.com/paper/2402.11131/) (2024)
- [k-fingerprinting: a Robust Scalable Website Fingerprinting Technique](https://mlsanity.com/paper/1509.00789/) (2015)
- [What Was Your Prompt? A Remote Keylogging Attack on AI Assistants](https://mlsanity.com/paper/2403.09751/) (2024)
- [Automated Website Fingerprinting through Deep Learning](https://mlsanity.com/paper/1708.06376/) (2017)
- [DeepCorr: Strong Flow Correlation Attacks on Tor Using Deep Learning](https://mlsanity.com/paper/1808.07285/) (2018)
- [The Loopix Anonymity System](https://mlsanity.com/paper/1703.00536/) (2017)

## Built on by (most cited first, 13 of 13)

- [Auditing Prompt Caching in Language Model APIs](https://mlsanity.com/paper/2502.07776/) (2025)
- [LLM Fingerprinting via Semantically Conditioned Watermarks](https://mlsanity.com/paper/2505.16723/) (2025)
- [When Speculation Spills Secrets: Side Channels via Speculative Decoding In LLMs](https://mlsanity.com/paper/2411.01076/) (2024)
- [Beyond Data Privacy: New Privacy Risks for Large Language Models](https://mlsanity.com/paper/2509.14278/) (2025)
- [Network-Level Prompt and Trait Leakage in Local Research Agents](https://mlsanity.com/paper/2508.20282/) (2025)
- [On the Promise for Assurance of Differentiable Neurosymbolic Reasoning Paradigms](https://mlsanity.com/paper/2502.08932/) (2025)
- [Whisper Leak: a side-channel attack on Large Language Models](https://mlsanity.com/paper/2511.03675/) (2025)
- [PrefixWall: Mitigating Prefix Caching Side Channels in Shared LLM Systems](https://mlsanity.com/paper/2603.10726/) (2026)
- [A Large-Scale Benchmark and Risk Assessment of Traffic Analysis Attacks on Cloud LLM Services](https://mlsanity.com/paper/2609.31877/) (2026)
- [Towards Anonymous Neural Network Inference](https://mlsanity.com/paper/2505.18398/) (2025)
- [Leaky Language Models: Stealing Architecture and Inference Optimizations via Per-Token Timing](https://mlsanity.com/paper/2607.20723/) (2026)
- [Sensitive-Topic Leakage Through LLM Routing Metadata: Measurement and Mitigation](https://mlsanity.com/paper/2610.09981/) (2026)
- [Speedbumps: Rejection Attacks on Speculative Decoding](https://mlsanity.com/paper/2610.10929/) (2026)

## Abstract (arXiv)

Scaling up language models has significantly increased their capabilities. But larger models are slower models, and so there is now an extensive body of work (e.g., speculative sampling or parallel decoding) that improves the (average case) efficiency of language model generation. But these techniques introduce data-dependent timing characteristics. We show it is possible to exploit these timing differences to mount a timing attack. By monitoring the (encrypted) network traffic between a victim user and a remote language model, we can learn information about the content of messages by noting when responses are faster or slower. With complete black-box access, on open source systems we show how it is possible to learn the topic of a user's conversation (e.g., medical advice vs. coding assistance) with 90%+ precision, and on production systems like OpenAI's ChatGPT and Anthropic's Claude we can distinguish between specific messages or infer the user's language. We further show that an active adversary can leverage a boosting attack to recover PII placed in messages (e.g., phone numbers or credit card numbers) for open source systems. We conclude with potential defenses and directions for future work.