# How Does Personalized Memory Shape LLM Behavior? Benchmarking Rational Preference Utilization in Personalized Assistants

arXiv: https://arxiv.org/abs/2601.16621 · HTML: https://mlsanity.com/paper/2601.16621/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 37 references that are in the map.
- First posted on arXiv 23 Jan 2026.

## Builds on (most cited first, 18 of 18)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Neural Collaborative Filtering](https://mlsanity.com/paper/1708.05031/) (2017)
- [Evaluating Very Long-Term Conversational Memory of LLM Agents](https://mlsanity.com/paper/2402.17753/) (2024)
- [LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory](https://mlsanity.com/paper/2410.10813/) (2024)
- [A Survey on the Memory Mechanism of Large Language Model based Agents](https://mlsanity.com/paper/2404.13501/) (2024)
- [All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Text](https://mlsanity.com/paper/2107.00061/) (2021)
- [MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent](https://mlsanity.com/paper/2507.02259/) (2025)
- [Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMs](https://mlsanity.com/paper/2502.09597/) (2025)
- [How Algorithmic Confounding in Recommendation Systems Increases Homogeneity and Decreases Utility](https://mlsanity.com/paper/1710.11214/) (2017)
- [MemoChat: Tuning LLMs to Use Memos for Consistent Long-Range Open-Domain Conversation](https://mlsanity.com/paper/2308.08239/) (2023)
- [Crafting Personalized Agents through Retrieval-Augmented Generation on Editable Memory Graphs](https://mlsanity.com/paper/2409.19401/) (2024)
- [Personalized LLM Response Generation with Parameterized Memory Injection](https://mlsanity.com/paper/2404.03565/) (2024)
- [From 1,000,000 Users to Every User: Scaling Up Personalized Preference for User-level Alignment](https://mlsanity.com/paper/2503.15463/) (2025)
- [PersonaBench: Evaluating AI Models on Understanding Personal Information through Accessing (Synthetic) Private User Data](https://mlsanity.com/paper/2502.20616/) (2025)
- [MemSim: A Bayesian Simulator for Evaluating Memory of LLM-based Personal Assistants](https://mlsanity.com/paper/2409.20163/) (2024)
- [Toward Multi-Session Personalized Conversation: A Large-Scale Dataset and Hierarchical Tree Framework for Implicit Reasoning](https://mlsanity.com/paper/2503.07018/) (2025)
- [AI-native Memory: A Pathway from LLMs Towards AGI](https://mlsanity.com/paper/2406.18312/) (2024)

## Built on by (most cited first, 9 of 9)

- [PERMA: Benchmarking Personalized Memory Agents via Event-Driven Preference and Realistic Task Environments](https://mlsanity.com/paper/2603.23231/) (2026)
- [MemCalib: Benchmarking and Optimizing Memory Use in LLM Agents](https://mlsanity.com/paper/2609.24259/) (2026)
- [Personalization Meets Safety:Mechanisms,Risks,and Mitigations in Personalized LLMs](https://mlsanity.com/paper/2606.09038/) (2026)
- [PersonaMem-v3: Toward Omni-Platform Personal Intelligence for Holistic User Understanding, Recommendation, and Agentic Tasks](https://mlsanity.com/paper/2608.21381/) (2026)
- [Evaluating the Hidden Costs of Personalization in Large Language Models](https://mlsanity.com/paper/2608.28833/) (2026)
- [PRAGMA: Evaluating Personalized Guidance with Memory Alignment in Lifelong Conversations](https://mlsanity.com/paper/2609.09664/) (2026)
- [Over-Personalization Is a Decision Failure: Generation-Induced Apply Bias in LLMs](https://mlsanity.com/paper/2609.34284/) (2026)
- [PairPref: When Should Memory Guide the Answer? A Benchmark for Contextual Preference Use](https://mlsanity.com/paper/2609.34526/) (2026)
- [PersEvalBench: Can LLM Evaluators Judge Contextually Appropriate Use of Persona Information?](https://mlsanity.com/paper/2610.19314/) (2026)

## Abstract (arXiv)

Large language model (LLM)-powered assistants have recently integrated memory mechanisms that record user preferences, leading to more personalized and user-aligned responses. However, irrelevant personalized memories are often introduced into the context, interfering with the LLM's intent understanding. To comprehensively investigate the dual effects of personalization, we develop RPEval, a benchmark comprising a personalized intent reasoning dataset and a multi-granularity evaluation protocol. RPEval reveals the widespread phenomenon of irrational personalization in existing LLMs and, through error pattern analysis, illustrates its negative impact on user experience. Finally, we introduce RP-Reasoner, which treats memory utilization as a pragmatic reasoning process, enabling the selective integration of personalized information. Experimental results demonstrate that our method significantly outperforms carefully designed baselines on RPEval, and resolves 80% of the bad cases observed in a large-scale commercial personalized assistant, highlighting the potential of pragmatic reasoning to mitigate irrational personalization. Our benchmark is publicly available at https://github.com/XueyangFeng/RPEval.