mlsanityData to 7 Oct 2026

Mem-Gallery: Benchmarking Multimodal Long-Term Conversational Memory for MLLM Agents

arXiv 2601.03515 ↗First posted 7 Jan 2026cs.CL cs.AIcited by 49

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

also in 1 more
Abstract (arXiv)

Long-term memory is a critical capability for multimodal large language model (MLLM) agents, particularly in conversational settings where information accumulates and evolves over time. However, existing benchmarks either evaluate multi-session memory in text-only conversations or assess multimodal understanding within localized contexts, failing to evaluate how multimodal memory is preserved, organized, and evolved across long-term conversational trajectories. Thus, we introduce Mem-Gallery, a new benchmark for evaluating multimodal long-term conversational memory in MLLM agents. Mem-Gallery features high-quality multi-session conversations grounded in both visual and textual information, with long interaction horizons and rich multimodal dependencies. Building on this dataset, we propose a systematic evaluation framework that assesses key memory capabilities along three functional dimensions: memory extraction and test-time adaptation, memory reasoning, and memory knowledge management. Extensive benchmarking across thirteen memory systems reveals several key findings, highlighting the necessity of explicit multimodal information retention and memory organization, the persistent limitations in memory reasoning and knowledge management, as well as the efficiency bottleneck of current models.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 27 of its 46 references that are in the map. First posted on arXiv 7 Jan 2026.

Builds on

27 of 46 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025MMRC: A Large-Scale Benchmark for Understanding Multimodal Large Language Model in Real-World Conversation17 Feb

Other references · 26

  1. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  2. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
  3. 2025A-MEM: Agentic Memory for LLM Agentscited by 1,000
  4. 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 800
  5. 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 656
  6. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 622
  7. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 480
  8. 2025Memory in the Age of AI Agentscited by 285
  9. 2025Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactionscited by 236
  10. 2025Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learningcited by 220
  11. 2022MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Textcited by 216
  12. 2021Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Docited by 178
14 more
  1. 2024GME: Improving Universal Multimodal Retrieval by Multimodal LLMscited by 166
  2. 2025LightMem: Lightweight and Efficient Memory-Augmented Generationcited by 163
  3. 2025Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memorycited by 129
  4. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 108
  5. 2022Long Time No See! Open-Domain Conversation with Long-Term Persona Memorycited by 107
  6. 2025How Memory Management Impacts LLM Agents: An Empirical Study of Experience-Following Behaviorcited by 95
  7. 2022MMDialog: A Large-scale Multi-turn Dialogue Dataset Towards Multi-modal Open-domain Conversationcited by 56
  8. 2023DialogBench: Evaluating LLMs as Human-like Dialogue Systemscited by 31
  9. 2023MPCHAT: Towards Multimodal Persona-Grounded Conversationcited by 24
  10. 2025UniversalRAG: Retrieval-Augmented Generation over Corpora of Diverse Modalities and Granularitiescited by 18
  11. 2025MemEngine: A Unified and Modular Library for Developing Advanced Memory of LLM-based Agentscited by 17
  12. 2025SABER: Switchable and Balanced Training for Efficient LLM Reasoningcited by 12
  13. 2025Agentic Learner with Grow-and-Refine Multimodal Semantic Memorycited by 11
  14. 2025AUGUSTUS: An LLM-Driven Multimodal Agent System with Contextualized User Memorycited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 49.

49 papers.

Sort:

By area:

  1. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 47
  2. 2026A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agentscited by 40
  3. 2026Code as Agent Harnesscited by 40
  4. 2026MemoryCD: Benchmarking Long-Context User Memory of LLM Agents for Lifelong Cross-Domain Personalizationcited by 15
  5. 2026Omni-SimpleMem: Autoresearch-Guided Discovery of Lifelong Multimodal Agent Memorycited by 13
  6. 2026Evo-Harness: Context-to-Harness Skill Compilation for Self-Evolving Agentscited by 12
  7. 2026EgoMemReason: A Memory-Driven Reasoning Benchmark for Long-Horizon Egocentric Video Understandingcited by 11
  8. 2026MemEye: A Visual-Centric Evaluation Framework for Multimodal Agent Memorycited by 10
  9. 2026Towards Multimodal Lifelong Understanding: A Dataset and Agentic Baselinecited by 8
  10. 2026Prune as You Generate: Online Rollout Pruning for Faster and Better RLVRcited by 8
  11. 2026AEL: Evolving Agent Harness in Open-Ended Environmentscited by 6
  12. 2026SMMBench: A Benchmark for Source-Distributed Multimodal Agent Memorycited by 5
37 more
  1. 2026MemTrace: Tracing and Attributing Errors in Large Language Model Memory Systemscited by 5
  2. 2026Generalizable Self-Evolving Memory for Automatic Prompt Optimizationcited by 4
  3. 2026Oblivion: Self-Adaptive Agentic Memory Control through Decay-Driven Activationcited by 3
  4. 2026FileGram: Grounding Agent Personalization in File-System Behavioral Tracescited by 3
  5. 2026PaperMind: Benchmarking Agentic Reasoning and Critique over Scientific Papers in Multimodal LLMscited by 3
  6. 2026MemLens: Benchmarking Multimodal Long-Term Memory in Large Vision-Language Modelscited by 3
  7. 2026H2HMem: A Multimodal Memory Benchmark for Agents in Human-Human Interactionscited by 3
  8. 2026Predict, Don't Iterate: Efficient Adaptive-Length Infilling for Diffusion Language Modelscited by 3
  9. 2026The Semantic Lifecycle in Embodied AI: Acquisition, Representation and Storage via Foundation Modelscited by 2
  10. 2026M3Exam: Benchmarking Multimodal Memory for Realistic User-Agent Interactionscited by 2
  11. 2026Less Context, More Accuracy: A Bi-Temporal Memory Engine for LLM Agents Where a Lean Retrieved Context Beats the Full Historycited by 2
  12. 2026ReContext: Recursive Evidence Replay as LLM Harness for Long-Context Reasoningcited by 2
  13. 2026PathMem: Toward Cognition-Aligned Memory Transformation for Pathology MLLMscited by 1
  14. 2026Task-Focused Memorization for Multimodal Agentscited by 1
  15. 2026M3Eval: Multi-Modal Memory Evaluation through Cognitively-Grounded Video Taskscited by 1
  16. 2026MemLeak: Diagnosing Information Leaks in Multimodal Agent Memorycited by 1
  17. 2026Do Agents Dream of False Memories? Black-box Visual Attacks on Long-term Memory in Multimodal AI Agentscited by 1
  18. 2026Agentic Evaluation of Copyright Law Compliancecited by 1
  19. 2026Beyond Retrieval: Analytic Memory for Multimodal Agentscited by 1
  20. 2026PMMC: Prospective Multimodal Memory Compilation for Long-Term LVLM Agentscited by 1
  21. 2026Multi-turn Conversational AI from Text to Multimodal Interaction: Data, Models, Evaluation, and Open Challengescited by 1
  22. 2026GraphMemix: Query-Aware Evidence Forests for Long-Term Multimodal Agent Memorycited by 1
  23. 2026Life-Bench: A Benchmark and Knowledge Graph Framework for Multimodal Personalization Beyond Concept Recognitioncited by 0
  24. 2026Personalizing Embodied Multimodal Large Language Model Agents over Long-term User Interactionscited by 0
  25. 2026ABot-AgentOS: A General Robotic Agent OS with Lifelong Multi-modal Memorycited by 0
  26. 2026MemTools: A Unified Research Framework for Interoperable Agent Memorycited by 0
  27. 2026MemArena: An Ego-Centric Benchmark for On-Device Agentic Personal Memory Assistants at Scalecited by 0
  28. 2026Towards a Formal Definition of Agent Memory: Basis, Span, Optimality, and the Sequential Memory Problemcited by 0
  29. 2026SciReC: Diagnostic Evaluation of Multimodal, Multi-Turn Relational Reasoning with Adaptive Interactioncited by 0
  30. 2026PolicyMem: Geometric Policy Memory for LLM Governancecited by 0
  31. 2026MIRAGE: How Conversation State Shapes Historical Evidence Use in Multimodal Personal Agentscited by 0
  32. 2026SIMLIFE: Pattern Understanding for Long-Horizon Human-Agent Partnershipcited by 0
  33. 2026EnSIMem: Entity-Structured Indexing for Long-Term Agent Memorycited by 0
  34. 2026EmbodiedMemory-Bench: Benchmarking Embodied Memory for Long-Horizon Embodied Taskscited by 0
  35. 2026Beyond Dyadic Memory: Interaction-Aware Multimodal Memory with Adaptive Agentic Retrieval for Multi-Party Spoken Conversationscited by 0
  36. 2026CUE-Mem: Benchmarking Long-Term User Memory via Implicit Cues in Multimodal Conversationscited by 0
  37. 2026MemFit: Efficient Long-Term Agentic Memorycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026WorldMemArena: Evaluating Multimodal Agent Memory Through Action-World Interaction
  2. 2026EvolMem: A Cognitive-Driven Benchmark for Multi-Session Dialogue Memory
  3. 2026AgentMemBench: A Systematic Benchmark for Evaluating Long-Term Memory Management Strategies in Conversational AI Agents
  4. 2026Memory Shot for Long-Term Dialogue ↗
  5. 2026MemOps: Benchmarking Lifecycle Memory Operations in Long-Horizon Conversations
  6. 2025From Single to Multi-Granularity: Toward Long-Term Memory Association and Selection of Conversational Agents
  7. 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
  8. 2026Memory in the LLM Era: Modular Architectures and Strategies in a Unified Framework