mlsanityData to 14 Oct 2026

"Nothing to See Here'': Unintended Disclosure through Revision Traces of LLM Deliverables

arXiv 2609.35408 ↗First posted 28 Sep 2026cs.CR cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language model (LLM) assistants increasingly help users draft content for third-party recipients. During private drafting, the user or the model may introduce an item and later remove or replace it. The model may remove the item from the intended content but reveal it again when stating the edit. We call such statements revision traces. For example, after a user removes the password before sharing a configuration file, the model may delete it but leave a comment saying, "Removed the password 'No****4!' as requested." A third-party recipient who sees only the delivered file can therefore recover the withdrawn password from the comment. In an in-the-wild analysis of three public conversation corpora, we identify 26,753 revision requests, of which 2,363 (8.8%) leave revision traces. We study them in greater depth under controlled conditions by introducing RevLeakBench, a benchmark of 100 tasks across five scenarios with a conversation track and an agent track. We measure trace occurrence, withdrawn-item recovery, trace position, and required-content retention. Across six models, about half of the deliverables in both tracks state the edit after a revocation, and a reader that sees only the deliverable can recover the withdrawn item from about 13% of them. Telling the model that its entire reply will be forwarded to the recipient still leaves revision traces in 36.4% of the deliverables. We compare prompt defenses and a delivery boundary, and propose an output-side filter that sharply reduces recovery with little loss of required content. We believe our work can benefit efforts to understand and mitigate unintended disclosure in LLM interactions.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 44 of its 58 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

44 of 58 references.

References · 44

  1. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  2. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  3. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 832
  4. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 629
  5. 2024TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Taskscited by 326
  6. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparisoncited by 258
  7. 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theorycited by 230
  8. 2023Beyond Memorization: Violating Privacy Via Inference with Large Language Modelscited by 175
  9. 2024PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Actioncited by 171
  10. 2025GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Taskscited by 158
  11. 2024Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Modelscited by 154
  12. 2025Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMscited by 148
32 more
  1. 2024InfiAgent-DABench: Evaluating Agents on Data Analysis Taskscited by 140
  2. 2024PLeak: Prompt Leaking Attacks against Large Language Model Applicationscited by 127
  3. 2024DiscoveryBench: Towards Data-Driven Discovery with Large Language Modelscited by 109
  4. 2025WritingBench: A Comprehensive Benchmark for Generative Writingcited by 106
  5. 2023LLM-Eval: Unified Multi-Dimensional Automatic Evaluation for Open-Domain Conversations with Large Language Modelscited by 101
  6. 2024AirGapAgent: Protecting Privacy-Conscious Conversational Agentscited by 94
  7. 2025AgentDAM: Privacy Leakage Evaluation for Autonomous Web Agentscited by 82
  8. 2023Did the Neurons Read your Book? Document-level Membership Inference for Large Language Modelscited by 56
  9. 2024Membership Inference Attacks Against In-Context Learningcited by 43
  10. 2024Private Attribute Inference from Images with Vision-Language Modelscited by 37
  11. 2024On the Privacy Risk of In-context Learningcited by 32
  12. 2023Deepfakes, Phrenology, Surveillance, and More! A Taxonomy of AI Privacy Riskscited by 31
  13. 2024WildFeedback: Aligning LLMs With In-situ User Interactions And Feedbackcited by 31
  14. 2025Leaky Thoughts: Large Reasoning Models Are Not Private Thinkerscited by 28
  15. 2025DRBench: A Realistic Benchmark for Enterprise Deep Researchcited by 22
  16. 2024Granular Privacy Control for Geolocation with Vision Language Modelscited by 21
  17. 2026How Your Credentials Are Leaked by LLM Agent Skills: An Empirical Studycited by 19
  18. 2024Raccoon: Prompt Extraction Benchmark of LLM-Integrated Applicationscited by 16
  19. 2025ShareChat: A Dataset of Chatbot Conversations in the Wildcited by 15
  20. 2026CI-Work: Benchmarking Contextual Integrity in Enterprise LLM Agentscited by 14
  21. 2023DevGPT: Studying Developer-ChatGPT Conversationscited by 12
  22. 2025Position: Contextual Integrity is Inadequately Applied to Language Modelscited by 11
  23. 2025SALT: Steering Activations towards Leakage-free Thinking in Chain of Thoughtcited by 11
  24. 2026Chain-of-Sanitized-Thoughts: Plugging PII Leakage in CoT of Large Reasoning Modelscited by 5
  25. 2026Capable but Careless: Do Computer-Use Agents Follow Contextual Integrity?cited by 3
  26. 2026Hidden Thoughts Are Not Secret: Reasoning Trace Exposure in LLMscited by 2
  27. 2026PrivacyAlign: Contextual Privacy Alignment for LLM Agentscited by 2
  28. 2026RedacBench: Can AI Erase Your Secrets?cited by 1
  29. 2026EarlySciRev: A Dataset of Early-Stage Scientific Revisions Extracted from LaTeX Writing Tracescited by 1
  30. 2026Hidden Secrets in the arXiv: Discovering, Analyzing, and Preventing Unintentional Information Disclosure in Source Files of Scientific Preprintscited by 1
  31. 2026Can You Keep a Secret? Involuntary Information Leakage in Language Model Writingcited by 1
  32. 2026To Add Is Machine, To Delete Is Human: Measuring and Mitigating Deletion Avoidance in LLM Code Editingcited by 1

Built on by

Not cited yet.