# "Ghost of the past": identifying and resolving privacy leakage from LLM's memory through proactive user interaction

arXiv: https://arxiv.org/abs/2410.14931 · HTML: https://mlsanity.com/paper/2410.14931/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Oct 2024.

## Built on by (most cited first, 10 of 10)

- [A Survey on Trustworthy LLM Agents: Threats and Countermeasures](https://mlsanity.com/paper/2503.09648/) (2025)
- [Position: Privacy Is Not Just Memorization!](https://mlsanity.com/paper/2510.01645/) (2025)
- [From tools to thieves: Measuring and understanding public perceptions of AI through crowdsourced metaphors](https://mlsanity.com/paper/2501.18045/) (2025)
- [Your LLM Agent Can Leak Your Data: Data Exfiltration via Backdoored Tool Use](https://mlsanity.com/paper/2604.05432/) (2026)
- [Mitigating Memorization in LLMs using Activation Steering](https://mlsanity.com/paper/2503.06040/) (2025)
- [Characterizing Unintended Consequences in Human-GUI Agent Collaboration for Web Browsing](https://mlsanity.com/paper/2505.09875/) (2025)
- [Safety in Embodied AI: A Survey of Risks, Attacks, and Defenses](https://mlsanity.com/paper/2605.02900/) (2026)
- [Automated Privacy Information Annotation in Large Language Model Interactions](https://mlsanity.com/paper/2505.20910/) (2025)
- [A Longitudinal Measurement of Privacy Policy Evolution for Large Language Models](https://mlsanity.com/paper/2511.21758/) (2025)
- [Secure Multifaceted-RAG for Enterprise: Hybrid Knowledge Retrieval with Security Filtering](https://mlsanity.com/paper/2504.13425/) (2025)

## Abstract (arXiv)

Memories, encompassing past inputs in context window and retrieval-augmented generation (RAG), frequently surface during human-LLM interactions, yet users are often unaware of their presence and the associated privacy risks. To address this, we propose MemoAnalyzer, a system for identifying, visualizing, and managing private information within memories. A semi-structured interview (N=40) revealed that low privacy awareness was the primary challenge, while proactive privacy control emerged as the most common user need. MemoAnalyzer uses a prompt-based method to infer and identify sensitive information from aggregated past inputs, allowing users to easily modify sensitive content. Background color temperature and transparency are mapped to inference confidence and sensitivity, streamlining privacy adjustments. A 5-day evaluation (N=36) comparing MemoAnalyzer with the default GPT setting and a manual modification baseline showed MemoAnalyzer significantly improved privacy awareness and protection without compromising interaction speed. Our study contributes to privacy-conscious LLM design, offering insights into privacy protection for Human-AI interactions.