mlsanityData to 7 Oct 2026

EMemBench: Interactive Benchmarking of Episodic Memory for VLM Agents

arXiv 2601.16690 ↗First posted 23 Jan 2026cs.CL cs.CVcited by 7

On the map: LLM agents and tool use → Agent memory systems

Abstract (arXiv)

We introduce EMemBench, a programmatic benchmark generator for evaluating long-term episodic memory of agents through interactive games. Rather than using a fixed set of questions, EMemBench generates questions from environment-grounded trajectories, covering both text-only and visual game environments. Each template computes verifiable ground truth from underlying game signals, with controlled answerability and balanced coverage over memory skills: single/multi-hop recall, induction, temporal, spatial, logical, and adversarial. We evaluate memory agents with strong LMs/VLMs as backbones, using in-context prompting as baselines. Across 15 text games and multiple visual seeds, results are far from saturated: induction and spatial reasoning are persistent bottlenecks, especially in visual settings. Persistent memory yields clear gains for open backbones on text games, but improvements are less consistent for VLM agents, suggesting that visually grounded episodic memory remains an open challenge. A human study further contextualizes the difficulty and interpretability of EMemBench.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Agent memory systems, one of 16 directions in LLM agents and tool use. Builds on 30 of its 45 references that are in the map. First posted on arXiv 23 Jan 2026.

Builds on

30 of 45 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning27 Aug
  2. 2025Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions7 Jul
  3. 2025MemBench: Towards More Comprehensive Evaluation on the Memory of LLM-based Agents20 Jun
  4. 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory28 Apr
  5. 2025A-MEM: Agentic Memory for LLM Agents17 Feb
  6. 2024AriGraph: Learning Knowledge Graph World Models with Episodic Memory for LLM Agents5 Jul
  7. 2024A Survey on the Memory Mechanism of Large Language Model based Agents21 Apr
  8. 2024Evaluating Very Long-Term Conversational Memory of LLM Agents27 Feb

Other references · 22

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  6. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  7. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  8. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
  9. 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 656
  10. 2019Scene Text Visual Question Answeringcited by 327
  11. 2025MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agentcited by 235
  12. 2021Benchmarking the Spectrum of Agent Capabilitiescited by 193
10 more
  1. 2024MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizationscited by 174
  2. 2019Interactive Fiction Games: A Colossal Adventurecited by 165
  3. 2025Evaluation and Benchmarking of LLM Agents: A Surveycited by 145
  4. 2023Document Understanding Dataset and Evaluation (DUDE)cited by 132
  5. 2025MemoryBench: A Benchmark for Memory and Continual Learning in LLM Systemscited by 56
  6. 2025Beyond a Million Tokens: Benchmarking and Enhancing Long-Term Memory in LLMscited by 52
  7. 2024PerLTQA: A Personal Long-Term Memory Dataset for Memory Classification, Retrieval, and Synthesis in Question Answeringcited by 34
  8. 2024Long Context vs. RAG for LLMs: An Evaluation and Revisitscited by 24
  9. 2025StoryBench: A Dynamic Benchmark for Evaluating Long-Term Memory with Multi Turnscited by 20
  10. 2023SPRING: Studying the Paper and Reasoning to Play Gamescited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Taskscited by 72
  2. 2026LongMemEval-V2: Evaluating Long-Term Agent Memory Toward Experienced Colleaguescited by 17
  3. 2026Towards Generalist Game Players: An Investigation of Foundation Models in the Game Multiversecited by 2
  4. 2026AI for Games in the Foundation Model Eracited by 2
  5. 2026S3Mem: Structured Spatiotemporal Scene-Event Memory for Long-Horizon Interactive Question Answeringcited by 1
  6. 2026DMV-Bench: Diagnosing Long-Horizon Multimodal Agents' Visual Memory with Incidental Cue Injectioncited by 1
  7. 2026Beyond the Current Observation: Evaluating Multimodal Large Language Models in Controllable Non-Markov Gamescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026AMA-Bench: Evaluating Long-Horizon Memory for Agentic Applications
  2. 2026MemEye: A Visual-Centric Evaluation Framework for Multimodal Agent Memory
  3. 2026Memory in the LLM Era: Modular Architectures and Strategies in a Unified Framework
  4. 2026EquiMem: Calibrating Shared Memory in Multi-Agent Debate via Game-Theoretic Equilibrium
  5. 2026EvoMemBench: Benchmarking Agent Memory from a Self-Evolving Perspective
  6. 2026AgentMemBench: A Systematic Benchmark for Evaluating Long-Term Memory Management Strategies in Conversational AI Agents
  7. 2026When Does Memory Help? A Cost-Aware Evaluation of Long-Term Memory in Tool-Using LLM Agents
  8. 2026PMMC: Prospective Multimodal Memory Compilation for Long-Term LVLM Agents