# Locomo-Plus: Beyond-Factual Cognitive Memory Evaluation Framework for LLM Agents

arXiv: https://arxiv.org/abs/2602.10715 · HTML: https://mlsanity.com/paper/2602.10715/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Placed in Agent memory systems, one of 16 directions in LLM agents and tool use.
- Builds on 21 of its 36 references that are in the map.
- First posted on arXiv 11 Feb 2026.

## Placed in

- LLM agents and tool use / [Agent memory systems](https://mlsanity.com/agents/agent-memory-systems/)

## Builds on (most cited first, 21 of 21)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [MemGPT: Towards LLMs as Operating Systems](https://mlsanity.com/paper/2310.08560/) (2023)
- [A-MEM: Agentic Memory for LLM Agents](https://mlsanity.com/paper/2502.12110/) (2025)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [Large Language Models Can Be Easily Distracted by Irrelevant Context](https://mlsanity.com/paper/2302.00093/) (2023)
- [Evaluating Very Long-Term Conversational Memory of LLM Agents](https://mlsanity.com/paper/2402.17753/) (2024)
- [CoQA: A Conversational Question Answering Challenge](https://mlsanity.com/paper/1808.07042/) (2018)
- [A Survey on the Memory Mechanism of Large Language Model based Agents](https://mlsanity.com/paper/2404.13501/) (2024)
- [From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge](https://mlsanity.com/paper/2411.16594/) (2024)
- [LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods](https://mlsanity.com/paper/2412.05579/) (2024)
- [Zep: A Temporal Knowledge Graph Architecture for Agent Memory](https://mlsanity.com/paper/2501.13956/) (2025)
- [True Few-Shot Learning with Language Models](https://mlsanity.com/paper/2105.11447/) (2021)
- [Beyond Goldfish Memory: Long-Term Open-Domain Conversation](https://mlsanity.com/paper/2107.07567/) (2021)
- [Why We Need New Evaluation Metrics for NLG](https://mlsanity.com/paper/1707.06875/) (2017)
- [Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judges](https://mlsanity.com/paper/2406.12624/) (2024)
- [From Human Memory to AI Memory: A Survey on Memory Mechanisms in the Era of LLMs](https://mlsanity.com/paper/2504.15965/) (2025)
- [Context Engineering 2.0: The Context of Context Engineering](https://mlsanity.com/paper/2510.26493/) (2025)
- [Flexible Prefrontal Control over Hippocampal Episodic Memory for Goal-Directed Generalization](https://mlsanity.com/paper/2503.02303/) (2025)

## Built on by (most cited first, 19 of 19)

- [Graph-Native Cognitive Memory for AI Agents: Formal Belief Revision Semantics for Versioned Memory Architectures](https://mlsanity.com/paper/2603.17244/) (2026)
- [Remember the Decision, Not the Description: A Rate-Distortion Framework for Agent Memory](https://mlsanity.com/paper/2605.10870/) (2026)
- [User as Code: Executable Memory for Personalized Agents](https://mlsanity.com/paper/2606.16707/) (2026)
- [A-MBER: Affective Memory Benchmark for Emotion Recognition](https://mlsanity.com/paper/2604.07017/) (2026)
- [SAGE: A Self-Evolving Agentic Graph-Memory Engine for Structure-Aware Associative Memory](https://mlsanity.com/paper/2605.12061/) (2026)
- [PersonaTree: Structured Lifecycle Memory for Person Understanding in LLM Agents](https://mlsanity.com/paper/2606.04780/) (2026)
- [When Users Don't Ask: Benchmarking Context-Driven Memory Retrieval in Conversational Agents](https://mlsanity.com/paper/2609.03467/) (2026)
- [Exploring Cross-Scenario Generality of Agentic Memory Systems: Diagnostics and a Strong Baseline](https://mlsanity.com/paper/2606.04315/) (2026)
- [T-Mem: Memory That Anticipates, Not Archives](https://mlsanity.com/paper/2606.15405/) (2026)
- [Keep It InMind: Benchmarking the Implicit-Association Blind Spot in Agent Memory](https://mlsanity.com/paper/2607.24368/) (2026)
- [JustMem: Just-Enough Memory Access for Long-Term Conversations](https://mlsanity.com/paper/2609.19877/) (2026)
- [FinLifeBench: Exhaustive Life-Event History and Financial-State Reconstruction from Longitudinal Banking Dialogue](https://mlsanity.com/paper/2609.01198/) (2026)
- [From Rollouts to Recipes: Self-Contained Post-Training for LLMs](https://mlsanity.com/paper/2609.01422/) (2026)
- [MeClear: Cooperative Game-Theoretic Attribution and Risk-Aware Memory Clearance for Long-Horizon LLM Agents](https://mlsanity.com/paper/2609.09115/) (2026)
- [AutoViewMem: Self-Configuring Orthogonal Views for Conversational Long-Term Memory](https://mlsanity.com/paper/2609.21940/) (2026)
- [Vestrum: Improving Agent Harnesses by Adapting Their Verification, Structure and Memory](https://mlsanity.com/paper/2609.33822/) (2026)
- [CAVE-Mem: Boundary-Aware Experience Validation for Memory Search](https://mlsanity.com/paper/2610.00238/) (2026)
- [Madeleine: Learning Involuntary Recall for Conversational Memory from Simulated Lives](https://mlsanity.com/paper/2610.01118/) (2026)
- [VitaBench: Memory Lifecycles Beyond Retrieval](https://mlsanity.com/paper/2610.14731/) (2026)

## Abstract (arXiv)

Long-term conversational memory is a core capability for LLM-based dialogue systems, yet existing benchmarks and evaluation protocols primarily focus on surface-level factual recall. In realistic interactions, appropriate responses often depend on implicit constraints such as user state, goals, or values that are not explicitly queried later. To evaluate this setting, we introduce LoCoMo-Plus, a benchmark for assessing cognitive memory under cue--trigger semantic disconnect, where models must retain and apply latent constraints across long conversational contexts. We further show that conventional string-matching metrics and explicit task-type prompting are misaligned with such scenarios, and propose a unified evaluation framework based on constraint consistency. Experiments across diverse backbone models, retrieval-based methods, and memory systems demonstrate that cognitive memory remains challenging and reveals failures not captured by existing benchmarks. Our code and evaluation framework are publicly available at: https://github.com/xjtuleeyf/Locomo-Plus.