# When Users Don't Ask: Benchmarking Context-Driven Memory Retrieval in Conversational Agents

arXiv: https://arxiv.org/abs/2609.03467 · HTML: https://mlsanity.com/paper/2609.03467/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Agent memory systems, one of 16 directions in LLM agents and tool use.
- Builds on 17 of its 65 references that are in the map.
- First posted on arXiv 3 Sep 2026.

## Placed in

- LLM agents and tool use / [Agent memory systems](https://mlsanity.com/agents/agent-memory-systems/)

## Builds on (most cited first, 17 of 17)

- [MemGPT: Towards LLMs as Operating Systems](https://mlsanity.com/paper/2310.08560/) (2023)
- [A-MEM: Agentic Memory for LLM Agents](https://mlsanity.com/paper/2502.12110/) (2025)
- [Evaluating Very Long-Term Conversational Memory of LLM Agents](https://mlsanity.com/paper/2402.17753/) (2024)
- [LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory](https://mlsanity.com/paper/2410.10813/) (2024)
- [Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions](https://mlsanity.com/paper/2507.05257/) (2025)
- [MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent](https://mlsanity.com/paper/2507.02259/) (2025)
- [LightMem: Lightweight and Efficient Memory-Augmented Generation](https://mlsanity.com/paper/2510.18866/) (2025)
- [Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scale](https://mlsanity.com/paper/2504.14225/) (2025)
- [Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMs](https://mlsanity.com/paper/2502.09597/) (2025)
- [PersonaMem-v2: Towards Personalized Intelligence via Learning Implicit User Personas and Agentic Memory](https://mlsanity.com/paper/2512.06688/) (2025)
- [Compress to Impress: Unleashing the Potential of Compressive Memory in Real-World Long-Term Conversations](https://mlsanity.com/paper/2402.11975/) (2024)
- [AMemGym: Interactive Memory Benchmarking for Assistants in Long-Horizon Conversations](https://mlsanity.com/paper/2603.01966/) (2026)
- [Locomo-Plus: Beyond-Factual Cognitive Memory Evaluation Framework for LLM Agents](https://mlsanity.com/paper/2602.10715/) (2026)
- [HorizonBench: Long-Horizon Personalization with Evolving Preferences](https://mlsanity.com/paper/2604.17283/) (2026)
- [MADial-Bench: Towards Real-world Evaluation of Memory-Augmented Dialogue Generation](https://mlsanity.com/paper/2409.15240/) (2024)
- [Memora: A Harmonic Memory Representation Balancing Abstraction and Specificity](https://mlsanity.com/paper/2602.03315/) (2026)
- [HiGMem: A Hierarchical and LLM-Guided Memory System for Long-Term Conversational Agents](https://mlsanity.com/paper/2604.18349/) (2026)

## Built on by (most cited first, 2 of 2)

- [MemFold: Learning Compact Soft Memory for Long-Context Personalization via On-Policy Optimization](https://mlsanity.com/paper/2609.36435/) (2026)
- [Madeleine: Learning Involuntary Recall for Conversational Memory from Simulated Lives](https://mlsanity.com/paper/2610.01118/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are increas- ingly deployed as long-horizon conversational agents, motivating growing interest in mem- ory systems. However, existing benchmarks primarily evaluate memory through QA-style probing rather than in-situ conversational usage. We introduce LOCOMO-CONV, a conversa- tional memory benchmark derived from Lo- CoMo with four query styles: dialog, implicit, counterfactual, and composed. Across five rep- resentative memory systems, we evaluate both retrieval recall and end-to-end response qual- ity. Our experiments show that conversational framing exposes substantial retrieval gaps over- looked by QA benchmarks, especially on im- plicit and composed queries, which multi-facet query rewriting narrows for raw-turn mem- ory but not abstractive memory. We further find that strong retrieval does not fully trans- late into response quality, and that implicit queries exhibit silent grounding, where mem- ory improves contextual grounding without ex- plicitly surfacing the gold fact. These results point to reasoning-based memory elaboration as a promising direction, and we release aux- iliary supportive_memory annotations captur- ing conversationally useful context beyond the original gold evidence.