# LUNAR: Benchmarking Personalized Large Language Models on UNiversal User BehAvioR Logs

arXiv: https://arxiv.org/abs/2608.05246 · HTML: https://mlsanity.com/paper/2608.05246/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 41 references that are in the map.
- First posted on arXiv 5 Aug 2026.

## Builds on (most cited first, 30 of 36)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory](https://mlsanity.com/paper/2504.19413/) (2025)
- [MemoryBank: Enhancing Large Language Models with Long-Term Memory](https://mlsanity.com/paper/2305.10250/) (2023)
- [LaMP: When Large Language Models Meet Personalization](https://mlsanity.com/paper/2304.11406/) (2023)
- [Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs](https://mlsanity.com/paper/2506.14245/) (2025)
- [LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals](https://mlsanity.com/paper/2411.10109/) (2024)
- [Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMs](https://mlsanity.com/paper/2502.09597/) (2025)
- [In Prospect and Retrospect: Reflective Memory Management for Long-term Personalized Dialogue Agents](https://mlsanity.com/paper/2503.08026/) (2025)
- [On Memory Construction and Retrieval for Personalized Conversational Agents](https://mlsanity.com/paper/2502.05589/) (2025)
- [Optimization Methods for Personalizing Large Language Models through Retrieval Augmentation](https://mlsanity.com/paper/2404.05970/) (2024)
- [Democratizing Large Language Models via Personalized Parameter-Efficient Fine-tuning](https://mlsanity.com/paper/2402.04401/) (2024)
- [Large Language Models Empowered Personalized Web Agents](https://mlsanity.com/paper/2410.17236/) (2024)
- [PersonaLens: A Benchmark for Personalization Evaluation in Conversational AI Assistants](https://mlsanity.com/paper/2506.09902/) (2025)
- [A Survey on Personalized Alignment -- The Missing Piece for Large Language Models in Real-World Applications](https://mlsanity.com/paper/2503.17003/) (2025)
- [LaMP-QA: A Benchmark for Personalized Long-form Question Answering](https://mlsanity.com/paper/2506.00137/) (2025)
- [Can LLM Agents Simulate Multi-Turn Human Behavior? Evidence from Real Online Customer Behavior Data](https://mlsanity.com/paper/2503.20749/) (2025)
- [OPeRA: A Dataset of Observation, Persona, Rationale, and Action for Evaluating LLMs on Human Online Shopping Behavior Simulation](https://mlsanity.com/paper/2506.05606/) (2025)
- [PersonaBench: Evaluating AI Models on Understanding Personal Information through Accessing (Synthetic) Private User Data](https://mlsanity.com/paper/2502.20616/) (2025)
- [RAP: Retrieval-Augmented Personalization for Multimodal Large Language Models](https://mlsanity.com/paper/2410.13360/) (2024)
- [OP-Bench: Benchmarking Over-Personalization for Memory-Augmented Personalized Conversational Agents](https://mlsanity.com/paper/2601.13722/) (2026)
- [Exploring the Potential of LLMs as Personalized Assistants: Dataset, Evaluation, and Analysis](https://mlsanity.com/paper/2506.01262/) (2025)
- [MemoryCD: Benchmarking Long-Context User Memory of LLM Agents for Lifelong Cross-Domain Personalization](https://mlsanity.com/paper/2603.25973/) (2026)
- [PersonaFeedback: A Large-scale Human-annotated Benchmark For Personalization](https://mlsanity.com/paper/2506.12915/) (2025)
- [Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History](https://mlsanity.com/paper/2602.17003/) (2026)
- [PrivacyBench: A Conversational Benchmark for Evaluating Privacy in Personalized AI](https://mlsanity.com/paper/2512.24848/) (2025)
- [Mem-PAL: Towards Memory-based Personalized Dialogue Assistants for Long-term User-Agent Interaction](https://mlsanity.com/paper/2511.13410/) (2025)
- [LifeSim: Long-Horizon User Life Simulator for Personalized Assistant Evaluation](https://mlsanity.com/paper/2603.12152/) (2026)
- [PersonaVLM: Long-Term Personalized Multimodal LLMs](https://mlsanity.com/paper/2604.13074/) (2026)
- [LifeBench: A Benchmark for Long-Horizon Multi-Source Memory](https://mlsanity.com/paper/2603.03781/) (2026)
- [Towards Natural Personalization: Evaluating Long-Horizon Preference Following in Personalized User-LLM Interactions](https://mlsanity.com/paper/2603.04191/) (2026)

## Built on by (most cited first, 2 of 2)

- [RealCompanion: Benchmarking Human Understanding from Reasoning over Longitudinal Real-World Conversations](https://mlsanity.com/paper/2610.01780/) (2026)
- [Vela: User Behavior as Supervision Sparks LLM Intelligence](https://mlsanity.com/paper/2610.14709/) (2026)

## Abstract (arXiv)

Existing personalized LLM benchmarks primarily rely on textual personas or isolated behavioral signals, providing limited evaluation of cross-domain behavioral personalization, where responses must be grounded in heterogeneous daily-life activities. To address this gap, we introduce LUNAR, the first benchmark for evaluating how LLMs personalize responses from longitudinal app interaction histories across universal daily-life domains, including clothing, food, housing, and mobility. To support scalable benchmark construction while mitigating data sparsity and privacy concerns, LUNAR uses a multi-stage coarse-to-fine synthesis pipeline grounded in real-world behavioral patterns. Fidelity analyses show closer alignment with real behavioral distributions than other synthetic benchmarks. Experiments on 19 mainstream LLMs show that access to behavioral logs is necessary but not sufficient for deep personalization: neither more context nor larger models guarantees better performance; effective personalization depends on selecting and integrating relevant evidence across domains. Direct retrieval of fine-grained behavioral records consistently outperforms compressed memory, while stronger personalization can come at the cost of privacy protection. These findings identify evidence selection, cross-domain integration, and privacy control as key challenges for personalized LLMs.