# A Personalized Conversational Benchmark: Towards Simulating Personalized Conversations

arXiv: https://arxiv.org/abs/2505.14106 · HTML: https://mlsanity.com/paper/2505.14106/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 43 references that are in the map.
- First posted on arXiv 20 May 2025.

## Builds on (most cited first, 21 of 21)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [LaMP: When Large Language Models Meet Personalization](https://mlsanity.com/paper/2304.11406/) (2023)
- [RedCaps: web-curated image-text data created by the people, for the people](https://mlsanity.com/paper/2111.11431/) (2021)
- [A Survey on Recent Advances in LLM-Based Multi-turn Dialogue Systems](https://mlsanity.com/paper/2402.18013/) (2024)
- [Personalization of Large Language Models: A Survey](https://mlsanity.com/paper/2411.00027/) (2024)
- [Conversational Health Agents: A Personalized LLM-Powered Agent Framework](https://mlsanity.com/paper/2310.02374/) (2023)
- [When Large Language Models Meet Personalization: Perspectives of Challenges and Opportunities](https://mlsanity.com/paper/2307.16376/) (2023)
- [MT-Eval: A Multi-Turn Capabilities Evaluation Benchmark for Large Language Models](https://mlsanity.com/paper/2401.16745/) (2024)
- [Teach LLMs to Personalize -- An Approach inspired by Writing Education](https://mlsanity.com/paper/2308.07968/) (2023)
- [Panoptic Scene Graph Generation with Semantics-Prototype Learning](https://mlsanity.com/paper/2307.15567/) (2023)
- [LoRe: Personalizing LLMs via Low-Rank Reward Modeling](https://mlsanity.com/paper/2504.14439/) (2025)
- [VLM2-Bench: A Closer Look at How Well VLMs Implicitly Link Explicit Matching Visual Cues](https://mlsanity.com/paper/2502.12084/) (2025)
- [Ruddit: Norms of Offensiveness for English Reddit Comments](https://mlsanity.com/paper/2106.05664/) (2021)
- [Beyond Prompts: Dynamic Conversational Benchmarking of Large Language Models](https://mlsanity.com/paper/2409.20222/) (2024)
- [Personalized Multimodal Large Language Models: A Survey](https://mlsanity.com/paper/2412.02142/) (2024)
- [xDial-Eval: A Multilingual Open-Domain Dialogue Evaluation Benchmark](https://mlsanity.com/paper/2310.08958/) (2023)
- [PersoBench: Benchmarking Personalized Response Generation in Large Language Models](https://mlsanity.com/paper/2410.03198/) (2024)
- [Personalized Graph-Based Retrieval for Large Language Models](https://mlsanity.com/paper/2501.02157/) (2025)
- [A Little Human Data Goes A Long Way](https://mlsanity.com/paper/2410.13098/) (2024)
- [REALM: A Dataset of Real-World LLM Use Cases](https://mlsanity.com/paper/2503.18792/) (2025)
- [Are Large Language Models In-Context Personalized Summarizers? Get an iCOPERNICUS Test Done!](https://mlsanity.com/paper/2410.00149/) (2024)

## Built on by (most cited first, 17 of 17)

- [A Survey on LLM-based Conversational User Simulation](https://mlsanity.com/paper/2604.24977/) (2026)
- [Toward Personalized LLM-Powered Agents: Foundations, Evaluation, and Future Directions](https://mlsanity.com/paper/2602.22680/) (2026)
- [HEART: A Unified Benchmark for Assessing Humans and LLMs in Emotional Support Dialogue](https://mlsanity.com/paper/2601.19922/) (2026)
- [PrefDisco: Benchmarking Proactive Personalized Reasoning](https://mlsanity.com/paper/2510.00177/) (2025)
- [ConvMix: A Mixed-Criteria Data Augmentation Framework for Conversational Dense Retrieval](https://mlsanity.com/paper/2508.04001/) (2025)
- [MambaTAD: When State-Space Models Meet Long-Range Temporal Action Detection](https://mlsanity.com/paper/2511.17929/) (2025)
- [Beyond the Embedding Bottleneck: Adaptive Retrieval-Augmented 3D CT Report Generation](https://mlsanity.com/paper/2603.15822/) (2026)
- [Memory Retrieval for Changing Preferences](https://mlsanity.com/paper/2606.02976/) (2026)
- [Pushing Forward Pareto Frontiers of Proactive Agents with Behavioral Agentic Optimization](https://mlsanity.com/paper/2602.11351/) (2026)
- [Cat-DPO: Category-Adaptive Safety Alignment](https://mlsanity.com/paper/2604.17299/) (2026)
- [F-GRPO: Factorized Group-Relative Policy Optimization for Unified Candidate Generation and Ranking](https://mlsanity.com/paper/2605.12995/) (2026)
- [CoReflect: A Reflective Co-Evolution Framework for Improving Conversational Evaluation](https://mlsanity.com/paper/2601.12208/) (2026)
- [TimeMM: Time-as-Operator Spectral Filtering for Dynamic Multimodal Recommendation](https://mlsanity.com/paper/2604.26247/) (2026)
- [Psi-Bench: Evaluating Persona-Sensitive Influencing in Persuasive Dialogues](https://mlsanity.com/paper/2606.02754/) (2026)
- [Agent Safety Is Action Alignment](https://mlsanity.com/paper/2606.28739/) (2026)
- [Benchmarking the Personalization Capabilities of Large Language Models](https://mlsanity.com/paper/2607.20471/) (2026)
- [What Happens During Autonomous Deep Research After the User Steps Away?](https://mlsanity.com/paper/2609.33509/) (2026)

## Abstract (arXiv)

We present PersonaConvBench, a large-scale benchmark for evaluating personalized reasoning and generation in multi-turn conversations with large language models (LLMs). Unlike existing work that focuses on either personalization or conversational structure in isolation, PersonaConvBench integrates both, offering three core tasks: sentence classification, impact regression, and user-centric text generation across ten diverse Reddit-based domains. This design enables systematic analysis of how personalized conversational context shapes LLM outputs in realistic multi-user scenarios. We benchmark several commercial and open-source LLMs under a unified prompting setup and observe that incorporating personalized history yields substantial performance improvements, including a 198 percent relative gain over the best non-conversational baseline in sentiment classification. By releasing PersonaConvBench with evaluations and code, we aim to support research on LLMs that adapt to individual styles, track long-term context, and produce contextually rich, engaging responses.