mlsanityData to 7 Oct 2026

Is He Extroverted? Identifying Missing Relevant Personas for Faithful User Simulation

arXiv 2602.15832 ↗First posted 3 Jan 2026cs.HC cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Existing user simulation approaches focus on generating user-like responses in dialogue. They often assume that the provided persona is sufficient for producing such responses, without verifying whether critical personas are supplied. This raises concerns about the validity of simulation results. To address this issue, we study the task of identifying persona dimensions (e.g., "whether the user is price-sensitive") that are relevant but missing in simulating a user's reply for a given dialogue context. We introduce PICQ-drama (constructed from TVShowGuess), a benchmark of context-aware choice questions, annotated with missing persona dimensions whose absence leads to ambiguous user choices. We further design diverse evaluation criteria for missing persona identification. Benchmarking leading LLMs on our PICQ-drama dataset demonstrates the feasibility of this task. Evaluation across diverse criteria, along with further analyses, reveals cognitive differences between LLMs and humans and highlights the distinct roles of different persona categories in shaping responses.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 15 of its 27 references that are in the map. First posted on arXiv 3 Jan 2026.

Builds on

15 of 27 references.

References · 15

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  4. 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 356
  5. 2023Character-LLM: A Trainable Agent for Role-Playingcited by 335
  6. 2021Beyond Goldfish Memory: Long-Term Open-Domain Conversationcited by 266
  7. 2024LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individualscited by 235
  8. 2023Better Zero-Shot Reasoning with Role-Play Promptingcited by 220
  9. 2024From Persona to Personalization: A Survey on Role-Playing Language Agentscited by 206
  10. 2024CharacterEval: A Chinese Benchmark for Role-Playing Conversational Agent Evaluationcited by 139
  11. 2024Large Language Models are Superpositions of All Characters: Attaining Arbitrary Role-play via Self-Alignmentcited by 104
  12. 2024Beyond Demographics: Aligning Role-playing LLM-based Agents Using Human Belief Networkscited by 39
3 more
  1. 2024Evaluating Character Understanding of Large Language Models via Character Profiling from Fictional Workscited by 38
  2. 2025OpenCharacter: Training Customizable Role-Playing LLMs with Large-Scale Synthetic Personascited by 25
  3. 2024How Well Can LLMs Echo Us? Evaluating AI Chatbots' Role-Play Ability with ECHOcited by 19

Built on by

Not cited yet.