# Re-Centering Humans in LLM Personalization

arXiv: https://arxiv.org/abs/2606.06614 · HTML: https://mlsanity.com/paper/2606.06614/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 47 references that are in the map.
- First posted on arXiv 4 Jun 2026.

## Builds on (most cited first, 30 of 35)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [HybridFlow: A Flexible and Efficient RLHF Framework](https://mlsanity.com/paper/2409.19256/) (2024)
- [LIMA: Less Is More for Alignment](https://mlsanity.com/paper/2305.11206/) (2023)
- [DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines](https://mlsanity.com/paper/2310.03714/) (2023)
- [Evaluating Very Long-Term Conversational Memory of LLM Agents](https://mlsanity.com/paper/2402.17753/) (2024)
- [LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory](https://mlsanity.com/paper/2410.10813/) (2024)
- [Automatic Prompt Optimization with "Gradient Descent" and Beam Search](https://mlsanity.com/paper/2305.03495/) (2023)
- [WildChat: 1M ChatGPT Interaction Logs in the Wild](https://mlsanity.com/paper/2405.01470/) (2024)
- [Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference](https://mlsanity.com/paper/2412.13663/) (2024)
- [LaMP: When Large Language Models Meet Personalization](https://mlsanity.com/paper/2304.11406/) (2023)
- [Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations](https://mlsanity.com/paper/2110.05719/) (2021)
- [Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scale](https://mlsanity.com/paper/2504.14225/) (2025)
- [Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMs](https://mlsanity.com/paper/2502.09597/) (2025)
- [Fine-Grained Analysis of Propaganda in News Articles](https://mlsanity.com/paper/1910.02517/) (2019)
- [Can LLM be a Personalized Judge?](https://mlsanity.com/paper/2406.11657/) (2024)
- [Cue-CoT: Chain-of-thought Prompting for Responding to In-depth Dialogue Questions with LLMs](https://mlsanity.com/paper/2305.11792/) (2023)
- [HYDRA: Model Factorization Framework for Black-Box LLM Personalization](https://mlsanity.com/paper/2406.02888/) (2024)
- [Flipping the Dialogue: Training and Evaluating User Language Models](https://mlsanity.com/paper/2510.06552/) (2025)
- [Persona-DB: Efficient Large Language Model Personalization for Response Prediction with Collaborative Data Refinement](https://mlsanity.com/paper/2402.11060/) (2024)
- [PersonaLens: A Benchmark for Personalization Evaluation in Conversational AI Assistants](https://mlsanity.com/paper/2506.09902/) (2025)
- [Creating General User Models from Computer Use](https://mlsanity.com/paper/2505.10831/) (2025)
- [Retrieval Augmented Generation with Collaborative Filtering for Personalized Text Generation](https://mlsanity.com/paper/2504.05731/) (2025)
- [CUPID: Evaluating Personalized and Contextualized Alignment of LLMs from Interactions](https://mlsanity.com/paper/2508.01674/) (2025)
- [Synthetic Socratic Debates: Examining Persona Effects on Moral Decision and Persuasion Dynamics](https://mlsanity.com/paper/2506.12657/) (2025)
- [Real or Robotic? Assessing Whether LLMs Accurately Simulate Qualities of Human Responses in Dialogue](https://mlsanity.com/paper/2409.08330/) (2024)
- [MultiSessionCollab: Learning User Preferences with Memory to Improve Long-Term Collaboration](https://mlsanity.com/paper/2601.02702/) (2026)
- [Measuring and Mitigating the Distributional Gap Between Real and Simulated User Behaviors](https://mlsanity.com/paper/2605.07847/) (2026)

## Built on by (most cited first, 1 of 1)

- [IdeaScientist: Orchestrating Agents for Grounded Scientific Ideation](https://mlsanity.com/paper/2610.04074/) (2026)

## Abstract (arXiv)

Despite growing interest, most evaluations of large language models' (LLMs') personalization abilities have relied on synthetic data. It remains unclear how well current personalization systems work for real users. In this paper, we study the gap in LLM personalization performance when using synthetic versus human data. We collect human conversations (550 conversations) and judgments across three stages of personalization: extracting user attributes from conversations (5,949 judgments), pairing relevant attributes with new prompts (11,919), and incorporating relevant attributes into a personalized response (1,101). Incorporating human data reveals system limitations at each stage. Models struggle to extract attributes from human conversations, disagree with human judgments on relevant attributes, and generate personalized responses that humans judge no better than generic responses (though that LLM judges widely rate as better). We introduce two lightweight training-based interventions that shift automated personalization evaluation closer to human data in our first two stages. However, in our third stage we find that learned reward models achieve only modest correlation with human ratings, suggesting that human-aligned personalization quality judgments are difficult to model directly. Our collected data provides a foundation for studying how models should extract, select, and incorporate user information in ways that humans find useful.