# PRAGMA: Evaluating Personalized Guidance with Memory Alignment in Lifelong Conversations

arXiv: https://arxiv.org/abs/2609.09664 · HTML: https://mlsanity.com/paper/2609.09664/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 29 references that are in the map.
- First posted on arXiv 9 Sep 2026.

## Builds on (most cited first, 21 of 21)

- [gpt-oss-120b & gpt-oss-20b Model Card](https://mlsanity.com/paper/2508.10925/) (2025)
- [A-MEM: Agentic Memory for LLM Agents](https://mlsanity.com/paper/2502.12110/) (2025)
- [Evaluating Very Long-Term Conversational Memory of LLM Agents](https://mlsanity.com/paper/2402.17753/) (2024)
- [LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory](https://mlsanity.com/paper/2410.10813/) (2024)
- [C-Pack: Packed Resources For General Chinese Embeddings](https://mlsanity.com/paper/2309.07597/) (2023)
- [RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval](https://mlsanity.com/paper/2401.18059/) (2024)
- [Precise Zero-Shot Dense Retrieval without Relevance Labels](https://mlsanity.com/paper/2212.10496/) (2022)
- [LightMem: Lightweight and Efficient Memory-Augmented Generation](https://mlsanity.com/paper/2510.18866/) (2025)
- [Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scale](https://mlsanity.com/paper/2504.14225/) (2025)
- [SimpleMem: Efficient Lifelong Memory for LLM Agents](https://mlsanity.com/paper/2601.02553/) (2026)
- [Hello Again! LLM-powered Personalized Agent for Long-term Dialogue](https://mlsanity.com/paper/2406.05925/) (2024)
- [In Prospect and Retrospect: Reflective Memory Management for Long-term Personalized Dialogue Agents](https://mlsanity.com/paper/2503.08026/) (2025)
- [Measuring Sycophancy of Language Models in Multi-turn Dialogues](https://mlsanity.com/paper/2505.23840/) (2025)
- [Toward Multi-Session Personalized Conversation: A Large-Scale Dataset and Hierarchical Tree Framework for Implicit Reasoning](https://mlsanity.com/paper/2503.07018/) (2025)
- [Efficient Context Selection for Long-Context QA: No Tuning, No Iteration, Just Adaptive-k](https://mlsanity.com/paper/2506.08479/) (2025)
- [Embodied Agents Meet Personalization: Investigating Challenges and Solutions Through the Lens of Memory Utilization](https://mlsanity.com/paper/2505.16348/) (2025)
- [Exploring the Potential of LLMs as Personalized Assistants: Dataset, Evaluation, and Analysis](https://mlsanity.com/paper/2506.01262/) (2025)
- [Convomem Benchmark: Why Your First 150 Conversations Don't Need RAG](https://mlsanity.com/paper/2511.10523/) (2025)
- [How Does Personalized Memory Shape LLM Behavior? Benchmarking Rational Preference Utilization in Personalized Assistants](https://mlsanity.com/paper/2601.16621/) (2026)
- [Privasis: Synthesizing the Largest "Public" Private Dataset from Scratch](https://mlsanity.com/paper/2602.03183/) (2026)
- [Inverse Knowledge Search over Verifiable Reasoning: Synthesizing a Scientific Encyclopedia from a Long Chains-of-Thought Knowledge Base](https://mlsanity.com/paper/2510.26854/) (2025)

## Abstract (arXiv)

Large language models (LLMs) are increasingly deployed as personalized assistants that interact with users over extended periods of time. As conversations grow longer, relying on full interaction histories becomes increasingly inefficient and unreliable: long contexts introduce substantial computational overhead, making it difficult for models to consistently identify and utilize the most relevant information for the current request. These challenges have motivated memory systems that structure and retrieve user-specific information. In realistic interactions, users often seek practical guidance such as recommendations, planning, and decision support. Unlike factual recall tasks, personalized guidance requires models to integrate information across multiple past conversations and reason about changing user preferences and experiences. However, existing conversational memory evaluations mainly focus on retrieval and factual recall. To study this challenge, we introduce PRAGMA, a benchmark for evaluating personalized guidance in long-term conversations. PRGAMA contains curated longitudinal conversation histories, evidence annotations, and guidance scenarios grounded in evolving user contexts and incorrect user assumptions. Experiments across retrieval systems, memory systems, and long-context models reveal that current systems struggle both to recover the appropriate conversational evidence and to effectively use it for personalized guidance. Our results highlight the need for memory architectures that support robust conversational retrieval and memory-grounded reasoning beyond evidence recall.