# Enjoy Your Talk: A Human-Centered Benchmark for Multi-Turn Dialogue with Decoupled User Simulation, Target Modeling, and Judging

arXiv: https://arxiv.org/abs/2607.10428 · HTML: https://mlsanity.com/paper/2607.10428/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 50 references that are in the map.
- First posted on arXiv 11 Jul 2026.

## Builds on (most cited first, 30 of 32)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains](https://mlsanity.com/paper/2406.12045/) (2024)
- [Large Language Models are not Fair Evaluators](https://mlsanity.com/paper/2305.17926/) (2023)
- [Personalizing Dialogue Agents: I have a dog, do you have pets too?](https://mlsanity.com/paper/1801.07243/) (2018)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [LLM Evaluators Recognize and Favor Their Own Generations](https://mlsanity.com/paper/2404.13076/) (2024)
- [GoEmotions: A Dataset of Fine-Grained Emotions](https://mlsanity.com/paper/2005.00547/) (2020)
- [LLMs Get Lost In Multi-Turn Conversation](https://mlsanity.com/paper/2505.06120/) (2025)
- [Scaling Synthetic Data Creation with 1,000,000,000 Personas](https://mlsanity.com/paper/2406.20094/) (2024)
- [MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMs](https://mlsanity.com/paper/2501.17399/) (2025)
- [Large language models that replace human participants can harmfully misportray and flatten identity groups](https://mlsanity.com/paper/2402.01908/) (2024)
- [CollabLLM: From Passive Responders to Active Collaborators](https://mlsanity.com/paper/2502.00640/) (2025)
- [PersonaMem-v2: Towards Personalized Intelligence via Learning Implicit User Personas and Agentic Memory](https://mlsanity.com/paper/2512.06688/) (2025)
- [Language Model Fine-Tuning on Scaled Survey Data for Predicting Distributions of Public Opinions](https://mlsanity.com/paper/2502.16761/) (2025)
- [A User Simulator for Task-Completion Dialogues](https://mlsanity.com/paper/1612.05688/) (2016)
- [A Comprehensive Analysis of the Effectiveness of Large Language Models as Automatic Dialogue Evaluators](https://mlsanity.com/paper/2312.15407/) (2023)
- [Consistently Simulating Human Personas with Multi-Turn Reinforcement Learning](https://mlsanity.com/paper/2511.00222/) (2025)
- [ChatBench: From Static Benchmarks to Human-AI Evaluation](https://mlsanity.com/paper/2504.07114/) (2025)
- [InteractiveOmni: A Unified Omni-modal Model for Audio-Visual Multi-turn Dialogue](https://mlsanity.com/paper/2510.13747/) (2025)
- [On Overcoming Miscalibrated Conversational Priors in LLM-based Chatbots](https://mlsanity.com/paper/2406.01633/) (2024)
- [Language Models in Dialogue: Conversational Maxims for Human-AI Interactions](https://mlsanity.com/paper/2403.15115/) (2024)
- [TrustJudge: Inconsistencies of LLM-as-a-Judge and How to Alleviate Them](https://mlsanity.com/paper/2509.21117/) (2025)
- [Improving Automatic Evaluation of Large Language Models (LLMs) in Biomedical Relation Extraction via LLMs-as-the-Judge](https://mlsanity.com/paper/2506.00777/) (2025)
- [A Desideratum for Conversational Agents: Capabilities, Challenges, and Future Directions](https://mlsanity.com/paper/2504.16939/) (2025)
- [MULTI-Bench: A Multi-Turn Interactive Benchmark for Assessing Emotional Intelligence ability of Spoken Dialogue Models](https://mlsanity.com/paper/2511.00850/) (2025)
- [One Battle After Another: Probing LLMs' Limits on Multi-Turn Instruction Following with a Benchmark Evolving Framework](https://mlsanity.com/paper/2511.03508/) (2025)
- [SkillAggregation: Reference-free LLM-Dependent Aggregation](https://mlsanity.com/paper/2410.10215/) (2024)
- [Evaluating and Mitigating LLM-as-a-judge Bias in Communication Systems](https://mlsanity.com/paper/2510.12462/) (2025)
- [EMPA: Evaluating Persona-Aligned Empathy as a Process](https://mlsanity.com/paper/2603.00552/) (2026)

## Built on by (most cited first, 1 of 1)

- [RELATE: An Evaluation Framework for measuring Relational Orientation of Large Language Models](https://mlsanity.com/paper/2610.09569/) (2026)

## Abstract (arXiv)

Evaluating large language models (LLMs) as multi-turn conversational partners requires probing capabilities that single-turn benchmarks miss: persona consistency, evolving intent tracking, emotional dynamics, and goal completion across many turns. We introduce EYT-Bench, a human-centered benchmark whose evaluation protocol is built around a decoupled three-party design: a persona-grounded user simulator, a target model evaluated on both intent perception and response generation, and an independent, configurable ensemble of LLM judges. Across 3,400 dialogues with 17 target models, EYT-Bench reveals four findings that previous benchmarks miss: (i) state-of-the-art closed and open-source models are statistically indistinguishable on subjective dimensions, but separate by up to 9x on objective intent-tracking; (ii) reasoning is a phase transition for objective tracking on long-context personas but is essentially flat on subjective scores; (iii) persona format strongly affects trajectory spread, FICR (final-intent completion rate) saturates above 0.95 on Nemotron-USA but ranges from 0.53 to 0.88 on PersonaMem-v2; and (iv) the warm-up effect is observed in 16 of 17 models.