# AudioRole: An Audio Dataset for Character Role-Playing in Large Language Models

arXiv: https://arxiv.org/abs/2509.23435 · HTML: https://mlsanity.com/paper/2509.23435/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 27 references that are in the map.
- First posted on arXiv 27 Sep 2025.

## Builds on (most cited first, 17 of 17)

- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Character-LLM: A Trainable Agent for Role-Playing](https://mlsanity.com/paper/2310.10158/) (2023)
- [GLM-4-Voice: Towards Intelligent and Human-Like End-to-End Spoken Chatbot](https://mlsanity.com/paper/2412.02612/) (2024)
- [VoiceBench: Benchmarking LLM-Based Voice Assistants](https://mlsanity.com/paper/2410.17196/) (2024)
- [AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Loss](https://mlsanity.com/paper/1905.05879/) (2019)
- [CharacterEval: A Chinese Benchmark for Role-Playing Conversational Agent Evaluation](https://mlsanity.com/paper/2401.01275/) (2024)
- [Meta Audiobox Aesthetics: Unified Automatic Quality Assessment for Speech, Music, and Sound](https://mlsanity.com/paper/2502.05139/) (2025)
- [CharacterChat: Learning towards Conversational AI with Personalized Social Support](https://mlsanity.com/paper/2308.10278/) (2023)
- [SocialBench: Sociality Evaluation of Role-Playing Conversational Agents](https://mlsanity.com/paper/2403.13679/) (2024)
- [Voice Conversion from Unaligned Corpora using Variational Autoencoding Wasserstein Generative Adversarial Networks](https://mlsanity.com/paper/1704.00849/) (2017)
- [Dialogue History Matters! Personalized Response Selectionin Multi-turn Retrieval-based Chatbots](https://mlsanity.com/paper/2103.09534/) (2021)
- [NAUTILUS: a Versatile Voice Cloning System](https://mlsanity.com/paper/2005.11004/) (2020)
- [VStyle: A Benchmark for Voice Style Adaptation with Spoken Instructions](https://mlsanity.com/paper/2509.09716/) (2025)
- [Towards General-Purpose Text-Instruction-Guided Voice Conversion](https://mlsanity.com/paper/2309.14324/) (2023)
- [Zero-shot Explainable Mental Health Analysis on Social Media by Incorporating Mental Scales](https://mlsanity.com/paper/2402.10948/) (2024)
- [SpeechRole: A Large-Scale Dataset and Benchmark for Evaluating Speech Role-Playing Agents](https://mlsanity.com/paper/2508.02013/) (2025)
- [HybridVC: Efficient Voice Style Conversion with Text and Audio Prompts](https://mlsanity.com/paper/2404.15637/) (2024)

## Built on by (most cited first, 1 of 1)

- [RoleBreak: Benchmarking Long-Horizon Role-Playing Robustness in Spoken Dialogue](https://mlsanity.com/paper/2609.16614/) (2026)

## Abstract (arXiv)

The creation of high-quality multimodal datasets remains fundamental for advancing role-playing capabilities in large language models (LLMs). While existing works predominantly focus on text-based persona simulation, Audio Role-Playing (ARP) presents unique challenges due to the need for synchronized alignment of semantic content and vocal characteristics. To address this gap, we propose AudioRole, a meticulously curated dataset from 13 TV series spanning 1K+ hours with 1M+ character-grounded dialogues, providing synchronized audio-text pairs annotated with speaker identities and contextual metadata. In addition, to demonstrate the effectiveness of the dataset, we introduced ARP-Eval, a dual-aspect evaluation framework that assesses both response quality and role fidelity. Empirical validation showing GLM-4-Voice trained on AudioRole (which we called ARP-Model) achieve an average Acoustic Personalization score of 0.31, significantly outperforming the original GLM-4-voice and the more powerful model MiniCPM-O-2.6, which specifically supports role-playing in one-shot scenarios. The ARP-Model also achieves a Content Personalization score of 0.36, surpassing the untrained original model by about 38% and maintaining the same level as MiniCPM-O-2.6. AudioRole features dialogues from over 115 main characters, 6 trained ARP-Models that role-play different characters, and evaluation protocols. Together, they provide an essential resource for advancing audio-grounded role-playing research.