A Large-Scale Benchmark and Risk Assessment of Traffic Analysis Attacks on Cloud LLM Services
Not on the map. Why?
Abstract (arXiv)
Cloud-based Large language model (LLM) services create a network-level traffic side channel that can expose model, prompt, and task behavior despite encryption. From packet sizes, directions, timing, and burst structure alone, a passive local observer can infer the serving model, the user's prompt category, and the task executed by a collaborative multi-agent system. Yet current evidence is fragmented across separate datasets and settings, limiting reproducibility and comparison. We present, to our knowledge, the first unified measurement study and public benchmark of encrypted LLM traffic across both user--LLM and multi-agent executions. The large-scale benchmark contains 60,000 user--LLM interactions across 10 models and 6 prompt categories, plus 2,838 multi-agent executions covering 10 task categories and two coordination topologies. Using only encrypted packet metadata, we assess the risk of traffic analysis attack by characterizing traffic signatures, identifying the features most associated with leakage, and testing robustness under prompt reformulation, decoding-temperature changes, larger candidate model sets, and partial traffic observation. Model fingerprinting achieves 97.7% balanced accuracy, prompt-category fingerprinting reaches 76.7% mean accuracy, and multi-agent task fingerprinting achieves up to 90.7% accuracy. Prompt reformulation weakens but does not remove model-specific leakage, and task fingerprints remain detectable even from a single agent's traffic.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 24 of its 35 references that are in the map. First posted on arXiv 25 Sep 2026.
Builds on
24 of 35 references.
References · 24
- 2020Measuring Massive Multitask Language Understandingcited by 7,963
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
- 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
- 2023LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Modelscited by 336
- 2025DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agentscited by 248
- 2024RouterBench: A Benchmark for Multi-LLM Routing Systemcited by 238
- 2025MultiAgentBench: Evaluating the Collaboration and Competition of LLM agentscited by 178
- 2024LLMmap: Fingerprinting For Large Language Modelscited by 58
- 2018Deep Fingerprinting: Undermining Website Fingerprinting Defenses with Deep Learningcited by 43
- 2026LLMRouterBench: A Massive Benchmark and Unified Framework for LLM Routingcited by 38
- 2015k-fingerprinting: a Robust Scalable Website Fingerprinting Techniquecited by 34
12 more
- 2024What Was Your Prompt? A Remote Keylogging Attack on AI Assistantscited by 26
- 2019Tik-Tok: The Utility of Packet Timing in Website Fingerprinting Attackscited by 15
- 2024Remote Timing Attacks on Efficient Language Model Inferencecited by 13
- 2023Realistic Website Fingerprinting By Augmenting Network Tracecited by 10
- 2025Exposing LLM User Privacy via Traffic Fingerprint Analysis: A Study of Privacy Risks in LLM Agent Interactionscited by 10
- 2026TeamBench: Evaluating Agent Coordination under Enforced Role Separationcited by 10
- 2025LLMs Have Rhythm: Fingerprinting Large Language Models Using Inter-Token Times and Network Traffic Analysiscited by 7
- 2025Network-Level Prompt and Trait Leakage in Local Research Agentscited by 5
- 2025NetEcho: From Real-World Streaming Side-Channels to Full LLM Conversation Recoverycited by 3
- 2025Whisper Leak: a side-channel attack on Large Language Modelscited by 3
- 2024A Measurement of Genuine Tor Traces for Realistic Website Fingerprintingcited by 2
- 2026PersonaFingerprint: Measuring Persona Inference on Modern Websites with LLM-Driven Browsingcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.