# BenchPreS: A Benchmark for Context-Aware Personalized Preference Selectivity of Persistent-Memory LLMs

arXiv: https://arxiv.org/abs/2603.16557 · HTML: https://mlsanity.com/paper/2603.16557/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 34 references that are in the map.
- First posted on arXiv 17 Mar 2026.

## Builds on (most cited first, 12 of 12)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Gorilla: Large Language Model Connected with Massive APIs](https://mlsanity.com/paper/2305.15334/) (2023)
- [LaMP: When Large Language Models Meet Personalization](https://mlsanity.com/paper/2304.11406/) (2023)
- [Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMs](https://mlsanity.com/paper/2502.09597/) (2025)
- [A Comprehensive Survey on Long Context Language Modeling](https://mlsanity.com/paper/2503.17407/) (2025)
- [Pearl: Personalizing Large Language Model Writing Assistants with Generation-Calibrated Retrievers](https://mlsanity.com/paper/2311.09180/) (2023)
- [AgentDAM: Privacy Leakage Evaluation for Autonomous Web Agents](https://mlsanity.com/paper/2503.09780/) (2025)
- [AdvancedIF: Rubric-Based Benchmarking and Reinforcement Learning for Advancing LLM Instruction Following](https://mlsanity.com/paper/2511.10507/) (2025)
- [Few-shot Personalization of LLMs with Mis-aligned Responses](https://mlsanity.com/paper/2406.18678/) (2024)
- [When Personalization Legitimizes Risks: Uncovering Safety Vulnerabilities in Personalized Dialogue Agents](https://mlsanity.com/paper/2601.17887/) (2026)
- [Understanding and Supporting Formal Email Exchange by Answering AI-Generated Questions](https://mlsanity.com/paper/2502.03804/) (2025)

## Built on by (most cited first, 10 of 10)

- [HorizonBench: Long-Horizon Personalization with Evolving Preferences](https://mlsanity.com/paper/2604.17283/) (2026)
- [Beyond Similarity: Trustworthy Memory Search for Personal AI Agents](https://mlsanity.com/paper/2606.06054/) (2026)
- [MemSyco-Bench: Benchmarking Sycophancy in Agent Memory](https://mlsanity.com/paper/2607.01071/) (2026)
- [MemCalib: Benchmarking and Optimizing Memory Use in LLM Agents](https://mlsanity.com/paper/2609.24259/) (2026)
- [ZenGen: Social Mind for LLMs](https://mlsanity.com/paper/2607.23740/) (2026)
- [Know It, Act on It: Investigating Memory Utilization in LLM Personalization](https://mlsanity.com/paper/2607.29433/) (2026)
- [Over-Personalization Is a Decision Failure: Generation-Induced Apply Bias in LLMs](https://mlsanity.com/paper/2609.34284/) (2026)
- [PairPref: When Should Memory Guide the Answer? A Benchmark for Contextual Preference Use](https://mlsanity.com/paper/2609.34526/) (2026)
- [RealCompanion: Benchmarking Human Understanding from Reasoning over Longitudinal Real-World Conversations](https://mlsanity.com/paper/2610.01780/) (2026)
- [Memadapter: Counterfactual Adaptation Against Memory-induced Sycophancy](https://mlsanity.com/paper/2610.05162/) (2026)

## Abstract (arXiv)

Large language models (LLMs) increasingly store user preferences in persistent memory to support personalization across interactions. However, in third-party communication settings governed by social and institutional norms, some user preferences may be inappropriate to apply. We introduce BenchPreS, which evaluates whether memory-based user preferences are appropriately applied or suppressed across communication contexts. Using two complementary metrics, Misapplication Rate (MR) and Appropriate Application Rate (AAR), we find even frontier LLMs struggle to apply preferences in a context-sensitive manner. Models with stronger preference adherence exhibit higher rates of over-application, and neither reasoning capability nor prompt-based defenses fully resolve this issue. These results suggest current LLMs treat personalized preferences as globally enforceable rules rather than as context-dependent normative signals.