Mem2ActBench: A Benchmark for Evaluating Long-Term Memory Utilization in Task-Oriented Autonomous Agents
On the map: LLM agents and tool use → Agent memory systems
Abstract (arXiv)
Large Language Model (LLM)-based agents are increasingly deployed for complex, tool-based tasks where long-term memory is critical to driving actions. Existing benchmarks, however, primarily test a angent's ability to passively retrieve isolated facts in response to explicit questions. They fail to evaluate the more crucial capability of actively applying memory to execute tasks. To address this gap, we introduce textscMem2ActBench, a benchmark for evaluating whether agents can proactively leverage long-term memory to execute tool-based actions by selecting appropriate tools and grounding their parameters. The benchmark simulates persistent assistant usage, where users mention the same topic across long, interrupted interactions and expect previously established preferences and task states to be implicitly applied. We build the dataset with an automated pipeline that merges heterogeneous sources (ToolACE, BFCL, Oasst1), resolves conflicts via consistency modeling, and synthesizes 2,029 sessions with 12 user--assistant--tool turns on average. From these memory chains, a reverse-generation method produces 400 tool-use tasks, with human evaluation confirming 91.3% are strongly memory-dependent. Experiments on seven memory frameworks show that current systems remain inadequate at actively utilizing memory for parameter grounding, highlighting the need for more effective approaches to evaluate and improve memory application in task execution.
Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). Placed in Agent memory systems, one of 16 directions in LLM agents and tool use. Builds on 12 of its 26 references that are in the map. First posted on arXiv 13 Jan 2026.
Builds on
12 of 26 references.
Closest ancestors · 3
Same direction, newest first
- 2025Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions7 Jul
- 2025A-MEM: Agentic Memory for LLM Agents17 Feb
- 2024Evaluating Very Long-Term Conversational Memory of LLM Agents27 Feb
Other references · 9
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
- 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
- 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,050
- 2022BERTopic: Neural topic modeling with a class-based TF-IDF procedurecited by 779
- 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 712
- 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 656
- 2023SCM: Enhancing Large Language Model with Self-Controlled Memory Frameworkcited by 98
- 2024From Isolated Conversations to Hierarchical Schemas: Dynamic Tree Memory Representation for LLMscited by 77
- 2024DialSim: A Dialogue Simulator for Evaluating Long-Term Multi-Party Dialogue Understanding of Conversational Agentscited by 19
Built on by
Citing papers per year: 2026: 30.
30 papers.
Sort:
By area:
- 2026MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Taskscited by 72
- 2026From Question Answering to Task Completion: A Survey on Agent System and Harness Designcited by 9
- 2026GateMem: Benchmarking Memory Governance in Multi-Principal Shared-Memory Agentscited by 8
- 2026PM-Bench: Evaluating Prospective Memory in LLM Agentscited by 6
- 2026π-Bench: Evaluating Proactive Personal Assistant Agents in Long-Horizon Workflowscited by 5
- 2026MemTxn: A Transaction Boundary for Source-Supported Updates and Complete-State Recovery in Agent Memorycited by 5
- 2026ClawVM: Harness-Managed Virtual Memory for Stateful Tool-Using LLM Agentscited by 4
- 2026MemTrace: Probing What Final Accuracy Misses in Long-Term Memorycited by 4
- 2026SafeCommit: Certifying When Memory-Grounded Agents May Safely Actcited by 4
- 2026From Agent Loops to Deterministic Graphs: Execution Lineage for Reproducible AI-Native Workcited by 2
- 2026MemArbiter: Decision-Time Memory Arbitration for Long-Horizon LLM Agentscited by 2
- 2026Remember, Verify, or Ask? Cross-Family Evaluation of Memory Commitment in LLM Agentscited by 2
18 more
- 2026SubtleMemory: A Benchmark for Fine-Grained Relational Memory Discrimination in Long-Horizon AI Agentscited by 1
- 2026When Personal Memory Has No Single Answer: Evaluating LLM Agents under Irreducible Conflictcited by 1
- 2026Agents in the Large: Perception-Centered Architecture for Persistent Agentscited by 1
- 2026Beyond Perplexity: A Behavioral Evaluation Framework for Deployment-Memory Claims in LLM Test-Time Trainingcited by 0
- 2026Agents Don't Just Agree, They Remember: Benchmarking Persistent Sycophancy in Self-Improving Personal Agentscited by 0
- 2026Beyond Memory Leaderboards: Evaluating Scientific Memory as Budgeted Context Restorationcited by 0
- 2026Ground Truth First: A Longitudinal Evaluation Instrument for Agent Memory, and the Tenure Crossover in Memory-Architecture Rankingscited by 0
- 2026UserToolBench: A User-Profile-Hidden Benchmark for Personalized Decision Making in Tool-Use LLMscited by 0
- 2026Towards a Formal Definition of Agent Memory: Basis, Span, Optimality, and the Sequential Memory Problemcited by 0
- 2026Handover of In-Context Learning State Across Session Boundariescited by 0
- 2026Explicit State Elicitation Is Not Enough: A Controlled Audit of Memory-Policy Classificationcited by 0
- 2026MemTrapBench: Benchmarking Cognitive Traps in LLM Memory Usecited by 0
- 2026Calibrating Criterion Revision in LLM Agents: Failure Modes and a Trace-Anchored Protocolcited by 0
- 2026MemCalib: Benchmarking and Optimizing Memory Use in LLM Agentscited by 0
- 2026DolphinBench: Mapping the Pareto Frontier of Agent Memorycited by 0
- 2026SCLATE: a Substrate for Continual-Learning Agent Training and Evaluationcited by 0
- 2026Contract Memory Compiler: Resolve, Then Traversecited by 0
- 2026Dude, Where's My State? Execution Information Requirements for Stateful Agentscited by 0
- 2026Dude, Where's My State? Execution Information Requirements for Stateful Agents26 Sep 2026
- 2026Contract Memory Compiler: Resolve, Then Traverse26 Sep 2026
- 2026SCLATE: a Substrate for Continual-Learning Agent Training and Evaluation26 Sep 2026
- 2026DolphinBench: Mapping the Pareto Frontier of Agent Memory21 Sep 2026
- 2026MemCalib: Benchmarking and Optimizing Memory Use in LLM Agents21 Sep 2026
- 2026Agents in the Large: Perception-Centered Architecture for Persistent Agents31 Aug 2026
- 2026Calibrating Criterion Revision in LLM Agents: Failure Modes and a Trace-Anchored Protocol21 Aug 2026
- 2026MemTrapBench: Benchmarking Cognitive Traps in LLM Memory Use20 Aug 2026
- 2026Remember, Verify, or Ask? Cross-Family Evaluation of Memory Commitment in LLM Agents20 Aug 2026
- 2026Explicit State Elicitation Is Not Enough: A Controlled Audit of Memory-Policy Classification18 Aug 2026
- 2026Handover of In-Context Learning State Across Session Boundaries14 Aug 2026
- 2026When Personal Memory Has No Single Answer: Evaluating LLM Agents under Irreducible Conflict14 Aug 2026
18 more
- 2026Towards a Formal Definition of Agent Memory: Basis, Span, Optimality, and the Sequential Memory Problem12 Aug 2026
- 2026UserToolBench: A User-Profile-Hidden Benchmark for Personalized Decision Making in Tool-Use LLMs10 Aug 2026
- 2026SafeCommit: Certifying When Memory-Grounded Agents May Safely Act4 Aug 2026
- 2026MemArbiter: Decision-Time Memory Arbitration for Long-Horizon LLM Agents3 Aug 2026
- 2026MemTxn: A Transaction Boundary for Source-Supported Updates and Complete-State Recovery in Agent Memory30 Jul 2026
- 2026Ground Truth First: A Longitudinal Evaluation Instrument for Agent Memory, and the Tenure Crossover in Memory-Architecture Rankings24 Jul 2026
- 2026Beyond Memory Leaderboards: Evaluating Scientific Memory as Budgeted Context Restoration18 Jul 2026
- 2026PM-Bench: Evaluating Prospective Memory in LLM Agents14 Jul 2026
- 2026Agents Don't Just Agree, They Remember: Benchmarking Persistent Sycophancy in Self-Improving Personal Agents12 Jul 2026
- 2026Beyond Perplexity: A Behavioral Evaluation Framework for Deployment-Memory Claims in LLM Test-Time Training1 Jul 2026
- 2026GateMem: Benchmarking Memory Governance in Multi-Principal Shared-Memory Agents17 Jun 2026
- 2026MemTrace: Probing What Final Accuracy Misses in Long-Term Memory15 Jun 2026
- 2026From Question Answering to Task Completion: A Survey on Agent System and Harness Design14 Jun 2026
- 2026SubtleMemory: A Benchmark for Fine-Grained Relational Memory Discrimination in Long-Horizon AI Agents4 Jun 2026
- 2026π-Bench: Evaluating Proactive Personal Assistant Agents in Long-Horizon Workflows14 May 2026
- 2026From Agent Loops to Deterministic Graphs: Execution Lineage for Reproducible AI-Native Work7 May 2026
- 2026ClawVM: Harness-Managed Virtual Memory for Stateful Tool-Using LLM Agents11 Apr 2026
- 2026MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks18 Feb 2026
- 2026MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Taskscited by 72
- 2026From Question Answering to Task Completion: A Survey on Agent System and Harness Designcited by 9
- 2026GateMem: Benchmarking Memory Governance in Multi-Principal Shared-Memory Agentscited by 8
- 2026PM-Bench: Evaluating Prospective Memory in LLM Agentscited by 6
- 2026π-Bench: Evaluating Proactive Personal Assistant Agents in Long-Horizon Workflowscited by 5
- 2026MemTxn: A Transaction Boundary for Source-Supported Updates and Complete-State Recovery in Agent Memorycited by 5
- 2026ClawVM: Harness-Managed Virtual Memory for Stateful Tool-Using LLM Agentscited by 4
- 2026MemTrace: Probing What Final Accuracy Misses in Long-Term Memorycited by 4
- 2026From Agent Loops to Deterministic Graphs: Execution Lineage for Reproducible AI-Native Workcited by 2
- 2026MemArbiter: Decision-Time Memory Arbitration for Long-Horizon LLM Agentscited by 2
- 2026Remember, Verify, or Ask? Cross-Family Evaluation of Memory Commitment in LLM Agentscited by 2
- 2026When Personal Memory Has No Single Answer: Evaluating LLM Agents under Irreducible Conflictcited by 1
12 most cited of 19 All 19 (JSON)
- 2026Contract Memory Compiler: Resolve, Then Traverse26 Sep 2026
- 2026MemCalib: Benchmarking and Optimizing Memory Use in LLM Agents21 Sep 2026
- 2026Agents in the Large: Perception-Centered Architecture for Persistent Agents31 Aug 2026
- 2026Calibrating Criterion Revision in LLM Agents: Failure Modes and a Trace-Anchored Protocol21 Aug 2026
- 2026Remember, Verify, or Ask? Cross-Family Evaluation of Memory Commitment in LLM Agents20 Aug 2026
- 2026When Personal Memory Has No Single Answer: Evaluating LLM Agents under Irreducible Conflict14 Aug 2026
- 2026UserToolBench: A User-Profile-Hidden Benchmark for Personalized Decision Making in Tool-Use LLMs10 Aug 2026
- 2026MemArbiter: Decision-Time Memory Arbitration for Long-Horizon LLM Agents3 Aug 2026
- 2026MemTxn: A Transaction Boundary for Source-Supported Updates and Complete-State Recovery in Agent Memory30 Jul 2026
- 2026Ground Truth First: A Longitudinal Evaluation Instrument for Agent Memory, and the Tenure Crossover in Memory-Architecture Rankings24 Jul 2026
- 2026Beyond Memory Leaderboards: Evaluating Scientific Memory as Budgeted Context Restoration18 Jul 2026
- 2026PM-Bench: Evaluating Prospective Memory in LLM Agents14 Jul 2026
12 newest of 19 All 19 (JSON)
- 2026SafeCommit: Certifying When Memory-Grounded Agents May Safely Actcited by 4
- 2026SubtleMemory: A Benchmark for Fine-Grained Relational Memory Discrimination in Long-Horizon AI Agentscited by 1
- 2026Beyond Perplexity: A Behavioral Evaluation Framework for Deployment-Memory Claims in LLM Test-Time Trainingcited by 0
- 2026Agents Don't Just Agree, They Remember: Benchmarking Persistent Sycophancy in Self-Improving Personal Agentscited by 0
- 2026Towards a Formal Definition of Agent Memory: Basis, Span, Optimality, and the Sequential Memory Problemcited by 0
- 2026Handover of In-Context Learning State Across Session Boundariescited by 0
- 2026Explicit State Elicitation Is Not Enough: A Controlled Audit of Memory-Policy Classificationcited by 0
- 2026MemTrapBench: Benchmarking Cognitive Traps in LLM Memory Usecited by 0
- 2026DolphinBench: Mapping the Pareto Frontier of Agent Memorycited by 0
- 2026SCLATE: a Substrate for Continual-Learning Agent Training and Evaluationcited by 0
- 2026Dude, Where's My State? Execution Information Requirements for Stateful Agentscited by 0
- 2026Dude, Where's My State? Execution Information Requirements for Stateful Agents26 Sep 2026
- 2026SCLATE: a Substrate for Continual-Learning Agent Training and Evaluation26 Sep 2026
- 2026DolphinBench: Mapping the Pareto Frontier of Agent Memory21 Sep 2026
- 2026MemTrapBench: Benchmarking Cognitive Traps in LLM Memory Use20 Aug 2026
- 2026Explicit State Elicitation Is Not Enough: A Controlled Audit of Memory-Policy Classification18 Aug 2026
- 2026Handover of In-Context Learning State Across Session Boundaries14 Aug 2026
- 2026Towards a Formal Definition of Agent Memory: Basis, Span, Optimality, and the Sequential Memory Problem12 Aug 2026
- 2026SafeCommit: Certifying When Memory-Grounded Agents May Safely Act4 Aug 2026
- 2026Agents Don't Just Agree, They Remember: Benchmarking Persistent Sycophancy in Self-Improving Personal Agents12 Jul 2026
- 2026Beyond Perplexity: A Behavioral Evaluation Framework for Deployment-Memory Claims in LLM Test-Time Training1 Jul 2026
- 2026SubtleMemory: A Benchmark for Fine-Grained Relational Memory Discrimination in Long-Horizon AI Agents4 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2026MemGym: a Long-Horizon Memory Environment for LLM Agents
- 2026AMA-Bench: Evaluating Long-Horizon Memory for Agentic Applications
- 2026EvoMemBench: Benchmarking Agent Memory from a Self-Evolving Perspective
- 2026MemReader: From Passive to Active Extraction for Long-Term Agent Memory
- 2026AgentMemBench: A Systematic Benchmark for Evaluating Long-Term Memory Management Strategies in Conversational AI Agents
- 2026MemToolAgent: Leveraging Memory for Tool Using Agents Based on Environment and User Feedback
- 2026VitaBench 2.0: Evaluating Personalized and Proactive Agents in Long-Term User Interactions
- 2026When Does Memory Help? A Cost-Aware Evaluation of Long-Term Memory in Tool-Using LLM Agents