mlsanityData to 14 Oct 2026

MemVerse: Multimodal Memory for Lifelong Learning Agents

arXiv 2512.03627 ↗First posted 3 Dec 2025cs.AIcited by 35

Not on the map. Why?

Abstract (arXiv)

Despite rapid progress in large-scale language and vision models, AI agents still suffer from a fundamental limitation: they cannot remember. Without reliable memory, agents catastrophically forget past experiences, struggle with long-horizon reasoning, and fail to operate coherently in multimodal or interactive environments. We introduce MemVerse, a model-agnostic, plug-and-play memory framework that bridges fast parametric recall with hierarchical retrieval-based memory, enabling scalable and adaptive multimodal intelligence. MemVerse maintains short-term memory for recent context while transforming raw multimodal experiences into structured long-term memories organized as hierarchical knowledge graphs. This design supports continual consolidation, adaptive forgetting, and bounded memory growth. To handle real-time demands, MemVerse introduces a periodic distillation mechanism that compresses essential knowledge from long-term memory into the parametric model, allowing fast, differentiable recall while preserving interpretability. Extensive experiments demonstrate that MemVerse significantly improves multimodal reasoning and continual learning efficiency, empowering agents to remember, adapt, and reason coherently across extended interactions.

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). Builds on 41 of its 76 references that are in the map. First posted on arXiv 3 Dec 2025.

Builds on

41 of 76 references.

References · 41

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  4. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  5. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
  6. 2025A-MEM: Agentic Memory for LLM Agentscited by 1,063
  7. 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 830
  8. 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 707
  9. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 634
  10. 2022InternVideo: General Video Foundation Models via Generative and Discriminative Learningcited by 447
  11. 2025Zep: A Temporal Knowledge Graph Architecture for Agent Memorycited by 400
  12. 2022X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrievalcited by 280
29 more
  1. 2023FireAct: Toward Language Agent Fine-tuningcited by 256
  2. 2017MemNet: A Persistent Memory Network for Image Restorationcited by 244
  3. 2025Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learningcited by 237
  4. 2024VideoAgent: A Memory-augmented Multimodal Agent for Video Understandingcited by 229
  5. 2023Unmasked Teacher: Towards Training-Efficient Video Foundation Modelscited by 199
  6. 2022MeMViT: Memory-Augmented Multiscale Vision Transformer for Efficient Long-Term Video Recognitioncited by 189
  7. 2025MIRIX: Multi-Agent Memory System for LLM-Based Agentscited by 173
  8. 2023VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Datasetcited by 169
  9. 2024MM-Embed: Universal Multimodal Retrieval with Multimodal LLMscited by 161
  10. 2020Embedding-based Retrieval in Facebook Searchcited by 156
  11. 2023mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Videocited by 154
  12. 2017Neural Episodic Controlcited by 151
  13. 2024Hello Again! LLM-powered Personalized Agent for Long-term Dialoguecited by 123
  14. 2024MemoRAG: Boosting Long Context Processing with Global Memory-Enhanced Retrieval Augmentationcited by 119
  15. 2024Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Modelscited by 116
  16. 2025MemGen: Weaving Generative Latent Memory for Self-Evolving Agentscited by 109
  17. 2024MegaPairs: Massive Data Synthesis For Universal Multimodal Retrievalcited by 84
  18. 2025FG-CLIP: Fine-Grained Visual and Textual Alignmentcited by 81
  19. 2023Agent Lumos: Unified and Modular Training for Open-Source Language Agentscited by 72
  20. 2024Multi-Modal Data-Efficient 3D Scene Understanding for Autonomous Drivingcited by 54
  21. 2024NoteLLM-2: Multimodal Large Representation Models for Recommendationcited by 44
  22. 2025Query-Focused Retrieval Heads Improve Long-Context Reasoning and Re-rankingcited by 31
  23. 2025Visual Instance-aware Prompt Tuningcited by 29
  24. 2025Aligning Vision to Language: Annotation-Free Multimodal Knowledge Graph Construction for Enhanced LLMs Reasoningcited by 19
  25. 2023MM-BigBench: Evaluating Multimodal Models on Multimodal Content Comprehension Taskscited by 18
  26. 2023Dual-Modal Attention-Enhanced Text-Video Retrieval with Triplet Partial Margin Contrastive Learningcited by 10
  27. 2025Evaluating Long-Term Memory for Long-Context Question Answeringcited by 8
  28. 2025Zero-shot Multimodal Document Retrieval via Cross-modal Question Generationcited by 6
  29. 2025Expertized Caption Auto-Enhancement for Video-Text Retrievalcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 30.

35 papers.

Sort:

By area:

  1. 2025Memory in the Age of AI Agentscited by 303
  2. 2026XSkill: Continual Learning from Experience and Skills in Multimodal Agentscited by 75
  3. 2026Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineeringcited by 74
  4. 2025Hindsight is 20/20: Building Agent Memory that Retains, Recalls, and Reflectscited by 44
  5. 2026A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agentscited by 44
  6. 2026Governing Evolving Memory in LLM Agents: Risks, Mechanisms, and the Stability and Safety Governed Memory (SSGM) Frameworkcited by 18
  7. 2026EvolveMem:Self-Evolving Memory Architecture via AutoResearch for LLM Agentscited by 15
  8. 2025TeleMem: Building Long-Term and Multimodal Memory for Agentic AIcited by 14
  9. 2026Omni-SimpleMem: Autoresearch-Guided Discovery of Lifelong Multimodal Agent Memorycited by 14
  10. 2025AI Meets Brain: Memory Systems from Cognitive Neuroscience to Autonomous Agentscited by 12
  11. 2026MementoGUI: Learning Agentic Multimodal Memory Control for Long-Horizon GUI Agentscited by 11
  12. 2026Position: Modular Memory is the Key to Continual Learning Agentscited by 8
23 more
  1. 2026One Model, All Roles: Multi-Turn, Multi-Agent Self-Play Reinforcement Learning for Conversational Social Intelligencecited by 7
  2. 2026SMMBench: A Benchmark for Source-Distributed Multimodal Agent Memorycited by 6
  3. 2026M3Exam: Benchmarking Multimodal Memory for Realistic User-Agent Interactionscited by 5
  4. 2026Hierarchical Memory Orchestration for Personalized Persistent Agentscited by 4
  5. 2026Persistent Visual Memory: Sustaining Perception for Deep Generation in LVLMscited by 4
  6. 2026V-Mem: Modality-Routed Retrieval for Long-Term Multimodal Agentic Memorycited by 4
  7. 2026HIMM: Human-Inspired Long-Term Memory Modeling for Embodied Exploration and Question Answeringcited by 3
  8. 2026FSFM: A Biologically-Inspired Framework for Selective Forgetting of Agent Memorycited by 3
  9. 2026Mem-π: Adaptive Memory through Learning When and What to Generatecited by 3
  10. 2026Mi-Memory: A Lifecycle Memory Framework for Personal AIcited by 3
  11. 2026Personal Visual Memory from Explicit and Implicit Evidencecited by 2
  12. 2026Do Agents Dream of False Memories? Black-box Visual Attacks on Long-term Memory in Multimodal AI Agentscited by 2
  13. 2025MIND: Multi-rationale INtegrated Discriminative Reasoning Framework for Multi-modal Large Modelscited by 1
  14. 2026Learning to Learn from Multimodal Experiencecited by 1
  15. 2026Task-Focused Memorization for Multimodal Agentscited by 1
  16. 2026HiMe: Hierarchical Embodied Memory for Long-Horizon Vision-Language-Action Controlcited by 1
  17. 2026TransMem: Transforming Hidden States into Memory for Large Language Modelscited by 1
  18. 2026Retrieved but Not Delivered: Multimodal Memory Delivery for Long-Term Agentscited by 1
  19. 2026Personalizing Embodied Multimodal Large Language Model Agents over Long-term User Interactionscited by 0
  20. 2026PathoSage: Towards Multi-Source Evidence Adjudication in Pathology via Experience-Aware Agentic Workflowcited by 0
  21. 2026EmbodiedMemory-Bench: Benchmarking Embodied Memory for Long-Horizon Embodied Taskscited by 0
  22. 2026MemPilot: Orchestrating On-Demand Multimodal Memory Curation for LLM Agentscited by 0
  23. 2026DSV-Mem: Evaluating Multimodal Memory in Professional Workflows for MLLM Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memory
  2. 2026MemWeaver: Weaving Hybrid Memories for Traceable Long-Horizon Agentic Reasoning
  3. 2026Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents
  4. 2026Dual-Grained Agent Memory and Shapley Context Attribution for Multimodal Agentic Learner
  5. 2025Agentic Learner with Grow-and-Refine Multimodal Semantic Memory
  6. 2026Advancing Multimodal Agent Reasoning with Long-Term Neuro-Symbolic Memory
  7. 2026MemGym: a Long-Horizon Memory Environment for LLM Agents
  8. 2025Mem-α: Learning Memory Construction via Reinforcement Learning