# PM-Bench: Evaluating Prospective Memory in LLM Agents arXiv: https://arxiv.org/abs/2607.12385 ยท HTML: https://mlsanity.com/paper/2607.12385/ ## Key facts - Cited by 7 papers in the map (data through 14 Oct 2026). - Placed in Agent memory systems, one of 16 directions in LLM agents and tool use. - Builds on 40 of its 55 references that are in the map. - First posted on arXiv 14 Jul 2026. ## Placed in - LLM agents and tool use / [Agent memory systems](https://mlsanity.com/agents/agent-memory-systems/) ## Builds on (most cited first, 30 of 40) - [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025) - [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023) - [MemGPT: Towards LLMs as Operating Systems](https://mlsanity.com/paper/2310.08560/) (2023) - [A-MEM: Agentic Memory for LLM Agents](https://mlsanity.com/paper/2502.12110/) (2025) - [OpenHands: An Open Platform for AI Software Developers as Generalist Agents](https://mlsanity.com/paper/2407.16741/) (2024) - [Evaluating Very Long-Term Conversational Memory of LLM Agents](https://mlsanity.com/paper/2402.17753/) (2024) - [LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory](https://mlsanity.com/paper/2410.10813/) (2024) - [BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents](https://mlsanity.com/paper/2504.12516/) (2025) - [Kimi K2: Open Agentic Intelligence](https://mlsanity.com/paper/2507.20534/) (2025) - [Beyond Goldfish Memory: Long-Term Open-Domain Conversation](https://mlsanity.com/paper/2107.07567/) (2021) - [Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions](https://mlsanity.com/paper/2507.05257/) (2025) - [MIRIX: Multi-Agent Memory System for LLM-Based Agents](https://mlsanity.com/paper/2507.07957/) (2025) - [SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualization](https://mlsanity.com/paper/2212.10465/) (2022) - [Memory OS of AI Agent](https://mlsanity.com/paper/2506.06326/) (2025) - [Prompting and Evaluating Large Language Models for Proactive Dialogues: Clarification, Target-guided, and Non-collaboration](https://mlsanity.com/paper/2305.13626/) (2023) - [In Prospect and Retrospect: Reflective Memory Management for Long-term Personalized Dialogue Agents](https://mlsanity.com/paper/2503.08026/) (2025) - [Recursive Language Models](https://mlsanity.com/paper/2512.24601/) (2025) - [Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models](https://mlsanity.com/paper/2601.07372/) (2026) - [MemBench: Towards More Comprehensive Evaluation on the Memory of LLM-based Agents](https://mlsanity.com/paper/2506.21605/) (2025) - [Qwen3-Coder-Next Technical Report](https://mlsanity.com/paper/2603.00729/) (2026) - [AMA-Bench: Evaluating Long-Horizon Memory for Agentic Applications](https://mlsanity.com/paper/2602.22769/) (2026) - [Conversation Chronicles: Towards Diverse Temporal and Relational Dynamics in Multi-Session Conversations](https://mlsanity.com/paper/2310.13420/) (2023) - [General Agentic Memory Via Deep Research](https://mlsanity.com/paper/2511.18423/) (2025) - [DeepPlanning: Benchmarking Long-Horizon Agentic Planning with Verifiable Constraints](https://mlsanity.com/paper/2601.18137/) (2026) - [Building Self-Evolving Agents via Experience-Driven Lifelong Learning: A Framework and Benchmark](https://mlsanity.com/paper/2508.19005/) (2025) - [REALTALK: A 21-Day Real-World Dataset for Long-Term Conversation](https://mlsanity.com/paper/2502.13270/) (2025) - [Robotouille: An Asynchronous Planning Benchmark for LLM Agents](https://mlsanity.com/paper/2502.05227/) (2025) - [Mem2ActBench: A Benchmark for Evaluating Long-Term Memory Utilization in Task-Oriented Autonomous Agents](https://mlsanity.com/paper/2601.19935/) (2026) - [MOSAIC: Modeling Social AI for Content Dissemination and Regulation in Multi-Agent Simulations](https://mlsanity.com/paper/2504.07830/) (2025) - [MPCHAT: Towards Multimodal Persona-Grounded Conversation](https://mlsanity.com/paper/2305.17388/) (2023) ## Built on by (most cited first, 7 of 7) - [Making Prospective Memory SLM-Shaped: Typed Intention Stores for Small-Model Agents](https://mlsanity.com/paper/2609.01272/) (2026) - [The Unreliable Progress Bar: Can LLM Agents Reliably Report Task Progress Throughout Execution?](https://mlsanity.com/paper/2609.08589/) (2026) - [How Strongly Should Task State Influence an LLM Agent?](https://mlsanity.com/paper/2609.25686/) (2026) - [When Should Agents Check External State? Budgeting Observations for Stored Intentions](https://mlsanity.com/paper/2609.37125/) (2026) - [ReLiveGym: Evaluating Long-Lived Agents over Weeks of Replayed Reality](https://mlsanity.com/paper/2610.00710/) (2026) - [Empty Commitments: When Agents Promise What They Cannot Deliver](https://mlsanity.com/paper/2610.01045/) (2026) - [Knowing the Store: What a Memory Backend Must Write Down Before an Agent Can Read It](https://mlsanity.com/paper/2610.04794/) (2026) ## Abstract (arXiv) A significant challenge in agentic AI is prospective memory: the ability to execute an intention at a specific future cue or state while other activities are ongoing. We introduce PM-Bench, a text-based benchmark for measuring prospective memory capabilities in modern LLM agents. Inspired by the Virtual Week paradigm from cognitive science, PM-Bench evaluates how well LLM agents maintain user intentions, execute delayed intentions, and monitor latent environment changes. Over the course of a simulated seven-day week, agents must continue an ongoing activity while deciding whether any deferred task is due. We compare eight state-of-the-art LLMs on PM-Bench under eight different agent configurations. PM-Bench proves challenging across all settings: the best method, a GPT-5.4 agent, reaches only 65.1% F1 score under our evaluation. Furthermore, no single strategy for improving prospective memory dominates across models. We release PM-Bench as a controlled testbed for diagnosing these failures and developing training or inference-time interventions that support reliable prospective behavior.