mlsanityData to 7 Oct 2026

What Was Your Prompt? A Remote Keylogging Attack on AI Assistants

arXiv 2403.09751 ↗First posted 14 Mar 2024cs.CR cs.AI cs.CLcited by 26

Not on the map. Why?

Abstract (arXiv)

AI assistants are becoming an integral part of society, used for asking advice or help in personal and confidential issues. In this paper, we unveil a novel side-channel that can be used to read encrypted responses from AI Assistants over the web: the token-length side-channel. We found that many vendors, including OpenAI and Microsoft, have this side-channel. However, inferring the content of a response from a token-length sequence alone proves challenging. This is because tokens are akin to words, and responses can be several sentences long leading to millions of grammatically correct sentences. In this paper, we show how this can be overcome by (1) utilizing the power of a large language model (LLM) to translate these sequences, (2) providing the LLM with inter-sentence context to narrow the search space and (3) performing a known-plaintext attack by fine-tuning the model on the target model's writing style. Using these methods, we were able to accurately reconstruct 29% of an AI assistant's responses and successfully infer the topic from 55% of them. To demonstrate the threat, we performed the attack on OpenAI's ChatGPT-4 and Microsoft's Copilot on both browser and API traffic.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 19 of its 42 references that are in the map. First posted on arXiv 14 Mar 2024.

Builds on

19 of 42 references.

References · 19

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  6. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,414
  7. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 825
  8. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  9. 2023Instruction Tuning for Large Language Models: A Surveycited by 617
  10. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 415
  11. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 380
  12. 2018Cache Telepathy: Leveraging Shared Resource Attacks to Learn DNN Architecturescited by 56
7 more
  1. 2018Stealing Neural Networks via Timing Side Channelscited by 42
  2. 2023Privacy Side Channels in Machine Learning Systemscited by 38
  3. 2018I Know What You See: Power Side-Channel Attack on Convolutional Neural Network Acceleratorscited by 26
  4. 2018Security Analysis of Deep Neural Networks Operating in the Presence of Cache Side-Channel Attackscited by 18
  5. 2020How to 0wn NAS in Your Spare Timecited by 10
  6. 2020Bicycle Attacks Considered Harmful: Quantifying the Damage of Widespread Password Length Leakagecited by 1
  7. 2024A Review and Comparison of AI Enhanced Side Channel Analysiscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 11; 2026: 8.

26 papers.

Sort:

By area:

  1. 2024AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathwayscited by 122
  2. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 44
  3. 2025Lessons From Red Teaming 100 Generative AI Productscited by 22
  4. 2025Auditing Prompt Caching in Language Model APIscited by 19
  5. 2024Remote Timing Attacks on Efficient Language Model Inferencecited by 13
  6. 2025Exposing LLM User Privacy via Traffic Fingerprint Analysis: A Study of Privacy Risks in LLM Agent Interactionscited by 9
  7. 2024A Jailbroken GenAI Model Can Cause Substantial Harm: GenAI-powered Applications are Vulnerable to PromptWarescited by 6
  8. 2024When Speculation Spills Secrets: Side Channels via Speculative Decoding In LLMscited by 6
  9. 2025Beyond Data Privacy: New Privacy Risks for Large Language Modelscited by 6
  10. 2025Network-Level Prompt and Trait Leakage in Local Research Agentscited by 5
  11. 2024Towards Action Hijacking of Large Language Model-based Agentcited by 4
  12. 2025Chain-of-Trigger: An Agentic Backdoor that Paradoxically Enhances Agentic Robustnesscited by 4
14 more
  1. 2024Memory Backdoor Attacks on Neural Networkscited by 3
  2. 2025Whisper Leak: a side-channel attack on Large Language Modelscited by 3
  3. 2025Spill The Beans: Exploiting CPU Cache Side-Channels to Leak Tokens from Large Language Modelscited by 2
  4. 2025Securing Educational LLMs: A Generalised Taxonomy of Attacks on LLMs and DREAD Risk Assessmentcited by 2
  5. 2026Rethinking Latency Denial-of-Service: Attacking the LLM Serving Framework, Not the Modelcited by 2
  6. 2025LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systemscited by 1
  7. 2026OptiLeak: Efficient Prompt Reconstruction via Reinforcement Learning in Multi-tenant LLM Servicescited by 1
  8. 2026The Proxy Knows Too Much: Sealing LLM API Routers with Attested TEEscited by 1
  9. 2026A Large-Scale Benchmark and Risk Assessment of Traffic Analysis Attacks on Cloud LLM Servicescited by 1
  10. 2025ASRJam: Human-Friendly AI Speech Jamming to Prevent Automated Phone Scamscited by 0
  11. 2026Conformal Prediction for Offensive Securitycited by 0
  12. 2026Detokenization Leaks: Reconstructing Local LLM Outputs From Cache Tracescited by 0
  13. 2026A Novel Semantic Manifold Alignment Attack against Embedding-to-Embedding Obfuscation in Privacy-Preserving LLMscited by 0
  14. 2026SparLeak: Privacy Leakage from Sparse Attention in LLM Inference on Shared GPUscited by 0