# Back in Style: A Sociolinguistic Approach to Authoring and Measuring Persona Fidelity in User Simulation

arXiv: https://arxiv.org/abs/2610.10988 · HTML: https://mlsanity.com/paper/2610.10988/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use.
- Builds on 23 of its 30 references that are in the map.
- First posted on arXiv 7 Oct 2026.

## Placed in

- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 23 of 23)

- [CharacterEval: A Chinese Benchmark for Role-Playing Conversational Agent Evaluation](https://mlsanity.com/paper/2401.01275/) (2024)
- [The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models](https://mlsanity.com/paper/2601.10387/) (2026)
- [Mind the Sim2Real Gap in User Simulation for Agentic Tasks](https://mlsanity.com/paper/2603.11245/) (2026)
- [Authorship Attribution in the Era of LLMs: Problems, Methodologies, and Challenges](https://mlsanity.com/paper/2408.08946/) (2024)
- [Lost in Simulation: LLM-Simulated Users are Unreliable Proxies for Human Users in Agentic Evaluations](https://mlsanity.com/paper/2601.17087/) (2026)
- [Consistently Simulating Human Personas with Multi-Turn Reinforcement Learning](https://mlsanity.com/paper/2511.00222/) (2025)
- [Quantifying and Optimizing Global Faithfulness in Persona-driven Role-playing](https://mlsanity.com/paper/2405.07726/) (2024)
- [TwinVoice: A Multi-dimensional Benchmark Towards Digital Twins via LLM Persona Simulation](https://mlsanity.com/paper/2510.25536/) (2025)
- [RecUserSim: A Realistic and Diverse User Simulator for Evaluating Conversational Recommender Systems](https://mlsanity.com/paper/2507.22897/) (2025)
- [PersonaEval: Are LLM Evaluators Human Enough to Judge Role-Play?](https://mlsanity.com/paper/2508.10014/) (2025)
- [The Chameleon's Limit: Investigating Persona Collapse and Homogenization in Large Language Models](https://mlsanity.com/paper/2604.24698/) (2026)
- [Can Authorship Attribution Models Distinguish Speakers in Speech Transcripts?](https://mlsanity.com/paper/2311.07564/) (2023)
- [MirrorBench: A Benchmark to Evaluate Conversational User-Proxy Agents for Human-Likeness](https://mlsanity.com/paper/2601.08118/) (2026)
- [Personality Expression Across Contexts: Linguistic and Behavioral Variation in LLM Agents](https://mlsanity.com/paper/2602.01063/) (2026)
- [Dialect vs Demographics: Quantifying LLM Bias from Implicit Linguistic Signals vs. Explicit User Profiles](https://mlsanity.com/paper/2604.21152/) (2026)
- [Eval4Sim: An Evaluation Framework for Persona Simulation](https://mlsanity.com/paper/2603.02876/) (2026)
- [MM-tau-p2: Persona-Adaptive Prompting for Robust Multi-Modal Agent Evaluation in Dual-Control Settings](https://mlsanity.com/paper/2603.09643/) (2026)
- [MUSE: Multi-Domain Chinese User Simulation via Self-Evolving Profiles and Rubric-Guided Alignment](https://mlsanity.com/paper/2604.13828/) (2026)
- [Simulated Customers Never Walk Away: Decision Fidelity of LLM User Simulators Measured Against Real Purchase Outcomes](https://mlsanity.com/paper/2606.20708/) (2026)
- [Explainable Disentangled Representation Learning for Generalizable Authorship Attribution in the Era of Generative AI](https://mlsanity.com/paper/2604.21300/) (2026)
- [LoCar: Localization-Aware Evaluation of In-Vehicle Assistants through Fine-Grained Sociolinguistic Control](https://mlsanity.com/paper/2605.21086/) (2026)
- [Can Multi-Agent LLMs Identify Their Peers? Stylometric Fingerprinting in Role-Constrained Political Analysis](https://mlsanity.com/paper/2606.09854/) (2026)
- [Role Steering of Language Models for Social Simulations](https://mlsanity.com/paper/2608.00023/) (2026)

## Abstract (arXiv)

As agentic systems gain commercial popularity, user simulators increasingly serve as measurement instrument for their evaluation. However, the fidelity of simulated users in comparison to real human users is generally low, and typically assessed by costly, subjective LLM judges. In this pilot study, we ask whether fidelity can instead be measured deterministically by treating a user persona sociolinguistically: as a social type that emerges from observable linguistic style, rather than one predicted by labels or descriptions a model must extrapolate into behaviour. We author personas as concrete stylistic rates, which lets us transfer two established, model-free instruments -- authorship-verification stylometry and lexicon-based content analysis -- as fidelity diagnostics. We A/B-test the sociolinguistic schema against a flat descriptive baseline across five task-oriented customer-service agents. Results show that the sociolinguistic schema improves both stylistic adherence and stylometric distinguishability for most of the tested models, with a caveat that persona style fidelity does not necessarily equal persona "naturalness". We argue that a sociolinguistic approach to persona design is a promising path towards more diverse and representative user personas, and that these metrics are most valuable in an error-attribution analysis, localizing where fidelity breaks down. This is a first step towards interventions that move user simulations closer to faithful renderings of diverse and variable linguistic outputs.