# MemEye: A Visual-Centric Evaluation Framework for Multimodal Agent Memory

arXiv: https://arxiv.org/abs/2605.15128 · HTML: https://mlsanity.com/paper/2605.15128/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use.
- Builds on 23 of its 46 references that are in the map.
- First posted on arXiv 14 May 2026.

## Placed in

- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 23 of 23)

- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory](https://mlsanity.com/paper/2504.19413/) (2025)
- [LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory](https://mlsanity.com/paper/2410.10813/) (2024)
- [Memory in the Age of AI Agents](https://mlsanity.com/paper/2512.13564/) (2025)
- [MIRIX: Multi-Agent Memory System for LLM-Based Agents](https://mlsanity.com/paper/2507.07957/) (2025)
- [SimpleMem: Efficient Lifelong Memory for LLM Agents](https://mlsanity.com/paper/2601.02553/) (2026)
- [Large Multimodal Agents: A Survey](https://mlsanity.com/paper/2402.15116/) (2024)
- [MMDU: A Multi-Turn Multi-Image Dialog Understanding Benchmark and Instruction-Tuning Dataset for LVLMs](https://mlsanity.com/paper/2406.11833/) (2024)
- [Mem-Gallery: Benchmarking Multimodal Long-Term Conversational Memory for MLLM Agents](https://mlsanity.com/paper/2601.03515/) (2026)
- [FluxMem: Adaptive Hierarchical Memory for Streaming Video Understanding](https://mlsanity.com/paper/2603.02096/) (2026)
- [M2A: Multimodal Memory Agent with Dual-Layer Hybrid Memory for Long-Term Personalized Interactions](https://mlsanity.com/paper/2602.07624/) (2026)
- [TwinVoice: A Multi-dimensional Benchmark Towards Digital Twins via LLM Persona Simulation](https://mlsanity.com/paper/2510.25536/) (2025)
- [According to Me: Long-Term Personalized Referential Memory QA](https://mlsanity.com/paper/2603.01990/) (2026)
- [Omni-SimpleMem: Autoresearch-Guided Discovery of Lifelong Multimodal Agent Memory](https://mlsanity.com/paper/2604.01007/) (2026)
- [R-WoM: Retrieval-augmented World Model For Computer-use Agents](https://mlsanity.com/paper/2510.11892/) (2025)
- [SafePred: A Predictive Guardrail for Computer-Using Agents via World Models](https://mlsanity.com/paper/2602.01725/) (2026)
- [HarmonyGuard: Toward Safety and Utility in Web Agents via Adaptive Policy Enhancement and Dual-Objective Optimization](https://mlsanity.com/paper/2508.04010/) (2025)
- [MultiVerse: A Multi-Turn Conversation Benchmark for Evaluating Large Vision and Language Models](https://mlsanity.com/paper/2510.16641/) (2025)
- [AEL: Evolving Agent Harness in Open-Ended Environments](https://mlsanity.com/paper/2604.21725/) (2026)
- [Avenir-Web: Human-Experience-Imitating Multimodal Web Agents with Mixture of Grounding Experts](https://mlsanity.com/paper/2602.02468/) (2026)
- [MMA: Multimodal Memory Agent](https://mlsanity.com/paper/2602.16493/) (2026)
- [Steering the Verifiability of Multimodal AI Hallucinations](https://mlsanity.com/paper/2604.06714/) (2026)
- [Individual Turing Test: A Case Study of LLM-based Simulation Using Longitudinal Personal Data](https://mlsanity.com/paper/2603.01289/) (2026)

## Built on by (most cited first, 14 of 14)

- [DMV-Bench: Diagnosing Long-Horizon Multimodal Agents' Visual Memory with Incidental Cue Injection](https://mlsanity.com/paper/2606.27499/) (2026)
- [Beyond Retrieval: Analytic Memory for Multimodal Agents](https://mlsanity.com/paper/2607.29440/) (2026)
- [MobileMem: Learning from a Year of Mobile Experiences](https://mlsanity.com/paper/2608.13606/) (2026)
- [Mitigating Identity Essentialism in LLM Agents with Longitudinal Life Trajectories](https://mlsanity.com/paper/2608.19621/) (2026)
- [GraphMemix: Query-Aware Evidence Forests for Long-Term Multimodal Agent Memory](https://mlsanity.com/paper/2608.26983/) (2026)
- [Retrieved but Not Delivered: Multimodal Memory Delivery for Long-Term Agents](https://mlsanity.com/paper/2609.32590/) (2026)
- [How do Humans Process AI-generated Hallucination Contents: a Neuroimaging Study](https://mlsanity.com/paper/2605.16953/) (2026)
- [Do GUI Agents Believe Their Eyes? Diagnosing State-Belief Reliance on Pixels versus Structure](https://mlsanity.com/paper/2607.04334/) (2026)
- [OpenPM: Auditable Point-in-Time Evaluation for LLM Portfolio-Management Agents](https://mlsanity.com/paper/2608.09988/) (2026)
- [ICM-Bench: Person-Level Identity Reasoning in Multimodal Agents with Long-Term Memory](https://mlsanity.com/paper/2609.04438/) (2026)
- [MemPilot: Orchestrating On-Demand Multimodal Memory Curation for LLM Agents](https://mlsanity.com/paper/2610.06830/) (2026)
- [Decide Before You Look: Learning Which Retrieved Memories Deserve Pixels](https://mlsanity.com/paper/2610.07984/) (2026)
- [DSV-Mem: Evaluating Multimodal Memory in Professional Workflows for MLLM Agents](https://mlsanity.com/paper/2610.08102/) (2026)
- [MemoWM: How World Models Change What Agents Need to Remember](https://mlsanity.com/paper/2610.10778/) (2026)

## Abstract (arXiv)

Long-term agent memory is increasingly multimodal, yet existing evaluations rarely test whether agents preserve the visual evidence needed for later reasoning. In prior work, many visually grounded questions can be answered using only captions or textual traces, allowing answers to be inferred without preserving the fine-grained visual evidence. Meanwhile, harder cases that require reasoning over changing visual states are largely absent. Therefore, we introduce MemEye, a framework that evaluates memory capabilities from two dimensions: one measures the granularity of decisive visual evidence (from scene-level to pixel-level evidence), and the other measures how retrieved evidence must be used (from single evidence to evolutionary synthesis). Under this framework, we construct a new benchmark across 8 life-scenario tasks, with ablation-driven validation gates for assessing answerability, shortcut resistance, visual necessity, and reasoning structure. By evaluating 13 memory methods across 4 VLM backbones, we show that current architectures still struggle to preserve fine-grained visual details and reason about state changes over time. Our findings show that long-term multimodal memory depends on evidence routing, temporal tracking, and detail extraction.