# MemLeak: Diagnosing Information Leaks in Multimodal Agent Memory

arXiv: https://arxiv.org/abs/2606.29788 · HTML: https://mlsanity.com/paper/2606.29788/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Agent memory systems, one of 16 directions in LLM agents and tool use.
- Builds on 27 of its 32 references that are in the map.
- First posted on arXiv 29 Jun 2026.

## Placed in

- LLM agents and tool use / [Agent memory systems](https://mlsanity.com/agents/agent-memory-systems/)

## Builds on (most cited first, 27 of 27)

- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Machine Unlearning](https://mlsanity.com/paper/1912.03817/) (2019)
- [Mass-Editing Memory in a Transformer](https://mlsanity.com/paper/2210.07229/) (2022)
- [TOFU: A Task of Fictitious Unlearning for LLMs](https://mlsanity.com/paper/2401.06121/) (2024)
- [Cognitive Architectures for Language Agents](https://mlsanity.com/paper/2309.02427/) (2023)
- [MUSE: Machine Unlearning Six-Way Evaluation for Language Models](https://mlsanity.com/paper/2407.06460/) (2024)
- [Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions](https://mlsanity.com/paper/2507.05257/) (2025)
- [Interpreting CLIP's Image Representation via Text-Based Decomposition](https://mlsanity.com/paper/2310.05916/) (2023)
- [Securing AI Agents with Information-Flow Control](https://mlsanity.com/paper/2505.23643/) (2025)
- [MemOS: An Operating System for Memory-Augmented Generation (MAG) in Large Language Models](https://mlsanity.com/paper/2505.22101/) (2025)
- [Position: LLM Unlearning Benchmarks are Weak Measures of Progress](https://mlsanity.com/paper/2410.02879/) (2024)
- [Collaborative Memory: Multi-User Memory Sharing in LLM Agents with Dynamic Access Control](https://mlsanity.com/paper/2505.18279/) (2025)
- [Mem-Gallery: Benchmarking Multimodal Long-Term Conversational Memory for MLLM Agents](https://mlsanity.com/paper/2601.03515/) (2026)
- [CLEAR: Character Unlearning in Textual and Visual Modalities](https://mlsanity.com/paper/2410.18057/) (2024)
- [Private Attribute Inference from Images with Vision-Language Models](https://mlsanity.com/paper/2404.10618/) (2024)
- [Ghost in the Agent: Redefining Information Flow Tracking for LLM Agents](https://mlsanity.com/paper/2604.23374/) (2026)
- [Forgetful but Faithful: A Cognitive Memory Architecture and Benchmark for Privacy-Aware Generative Agents](https://mlsanity.com/paper/2512.12856/) (2025)
- [An Information Theoretic Evaluation Metric For Strong Unlearning](https://mlsanity.com/paper/2405.17878/) (2024)
- [Unlearning Sensitive Information in Multimodal LLMs: Benchmark and Attack-Defense Evaluation](https://mlsanity.com/paper/2505.01456/) (2025)
- [Machine Unlearning under Retain-Forget Entanglement](https://mlsanity.com/paper/2603.26569/) (2026)
- [Graph-Native Cognitive Memory for AI Agents: Formal Belief Revision Semantics for Versioned Memory Architectures](https://mlsanity.com/paper/2603.17244/) (2026)
- [Cross-Modal Unlearning via Influential Neuron Path Editing in Multimodal Large Language Models](https://mlsanity.com/paper/2511.06793/) (2025)
- [Agentic Unlearning: When LLM Agent Meets Machine Unlearning](https://mlsanity.com/paper/2602.17692/) (2026)
- [SoK: Privacy Risks and Mitigations in Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2601.03979/) (2026)
- [Watch Out Your Album! On the Inadvertent Privacy Memorization in Multi-Modal Large Language Models](https://mlsanity.com/paper/2503.01208/) (2025)
- [SALMUBench: A Benchmark for Sensitive Association-Level Multimodal Unlearning](https://mlsanity.com/paper/2603.26316/) (2026)
- [Rethinking Visual Privacy: A Compositional Privacy Risk Framework for Severity Assessment with VLMs](https://mlsanity.com/paper/2603.21573/) (2026)

## Built on by (most cited first, 1 of 1)

- [MemSecBench: Tracking Agent Memory Poisoning from Persistence to Consequence and Repair](https://mlsanity.com/paper/2607.27080/) (2026)

## Abstract (arXiv)

When a multimodal AI agent is asked to forget a fact, current memory systems usually delete the text entry and report success. We find that the fact can remain recoverable from retained user images, including images tagged to entirely different facts, because VLMs use implicit visual cues at inference time. We introduce the Information Provenance Graph (IPG), a taxonomy that classifies memory representations by deletion affordance. The IPG reveals that deletion fails through multiple channels. Our benchmark, MemLeak, measures this across a deletion cascade: direct probing of deletion-capable systems yields <1%, but retained correlated text enables 18.3% recovery, and retained images enable 12.0% recovery (0.0% blind baseline, 0.3% FPR) -- with 47% of image leaks not text-recoverable. Content-aware semantic deletion reduces the image residual to 2.0%. The residual appears across multiple VLMs, a production memory system, and real Unsplash-licensed photographs. Dual-annotator human validation (kappa = 0.88) confirms judge reliability.