# MUSE: Multi-Domain Chinese User Simulation via Self-Evolving Profiles and Rubric-Guided Alignment

arXiv: https://arxiv.org/abs/2604.13828 · HTML: https://mlsanity.com/paper/2604.13828/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 19 of its 26 references that are in the map.
- First posted on arXiv 15 Apr 2026.

## Placed in

- LLM alignment / RLHF / preference optimisation / [LLM preference optimization methods](https://mlsanity.com/alignment/llm-preference-optimization-methods/)

## Builds on (most cited first, 19 of 19)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains](https://mlsanity.com/paper/2406.12045/) (2024)
- [WildChat: 1M ChatGPT Interaction Logs in the Wild](https://mlsanity.com/paper/2405.01470/) (2024)
- [Same Author or Just Same Topic? Towards Content-Independent Style Representations](https://mlsanity.com/paper/2204.04907/) (2022)
- [Let the LLMs Talk: Simulating Human-to-Human Conversational QA via Zero-Shot LLM-to-LLM Interactions](https://mlsanity.com/paper/2312.02913/) (2023)
- [Flipping the Dialogue: Training and Evaluating User Language Models](https://mlsanity.com/paper/2510.06552/) (2025)
- [A User Simulator for Task-Completion Dialogues](https://mlsanity.com/paper/1612.05688/) (2016)
- [The JDDC Corpus: A Large-Scale Multi-Turn Chinese Dialogue Dataset for E-commerce Customer Service](https://mlsanity.com/paper/1911.09969/) (2019)
- [Neural User Simulation for Corpus-based Policy Optimisation for Spoken Dialogue Systems](https://mlsanity.com/paper/1805.06966/) (2018)
- [Know You First and Be You Better: Modeling Human-Like User Simulators via Implicit Profiles](https://mlsanity.com/paper/2502.18968/) (2025)
- [A Sequence-to-Sequence Model for User Simulation in Spoken Dialogue Systems](https://mlsanity.com/paper/1607.00070/) (2016)
- [Consistently Simulating Human Personas with Multi-Turn Reinforcement Learning](https://mlsanity.com/paper/2511.00222/) (2025)
- [User Modeling for Task Oriented Dialogues](https://mlsanity.com/paper/1811.04369/) (2018)
- [NaturalConv: A Chinese Dialogue Dataset Towards Multi-turn Topic-driven Conversation](https://mlsanity.com/paper/2103.02548/) (2021)
- [ToolDial: Multi-turn Dialogue Generation Method for Tool-Augmented Language Models](https://mlsanity.com/paper/2503.00564/) (2025)
- [MUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use](https://mlsanity.com/paper/2508.18669/) (2025)
- [Examining Identity Drift in Conversations of LLM Agents](https://mlsanity.com/paper/2412.00804/) (2024)
- [A Desideratum for Conversational Agents: Capabilities, Challenges, and Future Directions](https://mlsanity.com/paper/2504.16939/) (2025)

## Built on by (most cited first, 2 of 2)

- [Controllable User Simulation](https://mlsanity.com/paper/2605.11519/) (2026)
- [Back in Style: A Sociolinguistic Approach to Authoring and Measuring Persona Fidelity in User Simulation](https://mlsanity.com/paper/2610.10988/) (2026)

## Abstract (arXiv)

User simulators are essential for the scalable training and evaluation of interactive AI systems. However, existing approaches often rely on shallow user profiling, struggle to maintain persona consistency over long interactions, and are largely limited to English or single-domain settings. We present MUSE, a multi-domain Chinese user simulation framework designed to generate human-like, controllable, and behaviorally consistent responses. First, we propose Iterative Profile Self-Evolution (IPSE), which gradually optimizes user profiles by comparing and reasoning discrepancies between simulated trajectories and real dialogue behaviors. We then apply Role-Reversal Supervised Fine-Tuning to improve local response realism and human-like expression. To enable fine-grained behavioral alignment, we further train a specialized rubric-based reward model and incorporate it into rubric-guided multi-turn reinforcement learning, which optimizes the simulator at the dialogue level and enhances long-horizon behavioral consistency. Experiments show that MUSE consistently outperforms strong baselines in both utterance-level and session-level evaluations, generating responses that are more realistic, coherent, and persona-consistent over extended interactions.