MUSE: Multi-Domain Chinese User Simulation via Self-Evolving Profiles and Rubric-Guided Alignment
On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods
Abstract (arXiv)
User simulators are essential for the scalable training and evaluation of interactive AI systems. However, existing approaches often rely on shallow user profiling, struggle to maintain persona consistency over long interactions, and are largely limited to English or single-domain settings. We present MUSE, a multi-domain Chinese user simulation framework designed to generate human-like, controllable, and behaviorally consistent responses. First, we propose Iterative Profile Self-Evolution (IPSE), which gradually optimizes user profiles by comparing and reasoning discrepancies between simulated trajectories and real dialogue behaviors. We then apply Role-Reversal Supervised Fine-Tuning to improve local response realism and human-like expression. To enable fine-grained behavioral alignment, we further train a specialized rubric-based reward model and incorporate it into rubric-guided multi-turn reinforcement learning, which optimizes the simulator at the dialogue level and enhances long-horizon behavioral consistency. Experiments show that MUSE consistently outperforms strong baselines in both utterance-level and session-level evaluations, generating responses that are more realistic, coherent, and persona-consistent over extended interactions.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 19 of its 26 references that are in the map. First posted on arXiv 15 Apr 2026.
Builds on
19 of 26 references.
Closest ancestors · 1
Same direction, newest first
Other references · 18
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
- 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 598
- 2022Same Author or Just Same Topic? Towards Content-Independent Style Representationscited by 83
- 2023Let the LLMs Talk: Simulating Human-to-Human Conversational QA via Zero-Shot LLM-to-LLM Interactionscited by 70
- 2016A User Simulator for Task-Completion Dialoguescited by 56
- 2025Flipping the Dialogue: Training and Evaluating User Language Modelscited by 50
- 2019The JDDC Corpus: A Large-Scale Multi-Turn Chinese Dialogue Dataset for E-commerce Customer Servicecited by 47
- 2018Neural User Simulation for Corpus-based Policy Optimisation for Spoken Dialogue Systemscited by 42
- 2016A Sequence-to-Sequence Model for User Simulation in Spoken Dialogue Systemscited by 39
- 2025Know You First and Be You Better: Modeling Human-Like User Simulators via Implicit Profilescited by 38
- 2018User Modeling for Task Oriented Dialoguescited by 36
6 more
- 2021NaturalConv: A Chinese Dialogue Dataset Towards Multi-turn Topic-driven Conversationcited by 35
- 2025Consistently Simulating Human Personas with Multi-Turn Reinforcement Learningcited by 35
- 2025ToolDial: Multi-turn Dialogue Generation Method for Tool-Augmented Language Modelscited by 34
- 2025MUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool usecited by 30
- 2024Examining Identity Drift in Conversations of LLM Agentscited by 25
- 2025A Desideratum for Conversational Agents: Capabilities, Challenges, and Future Directionscited by 9
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026TRACER: Trajectory-Aligned Learning for Multi-Turn User Simulation
- 2025Teaching Language Models to Evolve with Users: Dynamic Profile Modeling for Personalized Alignment
- 2026UserProxyBench: Evaluating LLM User Simulators for Agent Benchmarks and Training
- 2026PersonaForge: Realistic Multi-Turn User Simulation for Agentic Systems
- 2026Beyond Cooperative Simulators: Generating Realistic User Personas for Robust Evaluation of LLM Agents
- 2026HumanLM: Simulating Users with State Alignment Beats Response Imitation
- 2024Sparse Rewards Can Self-Train Dialogue Agents
- 2025Aligning Large Language Models via Fully Self-Synthetic Data