# Playing with Voices: Tabletop Role-Playing Game Recordings as a Diarization Challenge

arXiv: https://arxiv.org/abs/2502.12714 · HTML: https://mlsanity.com/paper/2502.12714/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 43 references that are in the map.
- First posted on arXiv 18 Feb 2025.

## Builds on (most cited first, 16 of 16)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [CHiME-6 Challenge:Tackling Multispeaker Speech Recognition for Unsegmented Recordings](https://mlsanity.com/paper/2004.09249/) (2020)
- [A Review of Speaker Diarization: Recent Advances with Deep Learning](https://mlsanity.com/paper/2101.09624/) (2021)
- [Powerset multi-class cross entropy loss for neural speaker diarization](https://mlsanity.com/paper/2310.13025/) (2023)
- [The People's Speech: A Large-Scale Diverse English Speech Recognition Dataset for Commercial Usage](https://mlsanity.com/paper/2111.09344/) (2021)
- [Wespeaker: A Research and Production oriented Speaker Embedding Learning Toolkit](https://mlsanity.com/paper/2210.17016/) (2022)
- [Spot the conversation: speaker diarisation in the wild](https://mlsanity.com/paper/2007.01216/) (2020)
- [The Third DIHARD Diarization Challenge](https://mlsanity.com/paper/2012.01477/) (2020)
- [Dungeons and Dragons as a Dialog Challenge for Artificial Intelligence](https://mlsanity.com/paper/2210.07109/) (2022)
- [Speaker Diarization with Lexical Information](https://mlsanity.com/paper/2004.06756/) (2020)
- [Telling Stories through Multi-User Dialogue by Modeling Character Relations](https://mlsanity.com/paper/2105.15054/) (2021)
- [PaCo: Preconditions Attributed to Commonsense Knowledge](https://mlsanity.com/paper/2104.08712/) (2021)
- [The timing bottleneck: Why timing and overlap are mission-critical for conversational user interfaces, speech recognition and dialogue systems](https://mlsanity.com/paper/2307.15493/) (2023)
- [USTC-NELSLIP System Description for DIHARD-III Challenge](https://mlsanity.com/paper/2103.10661/) (2021)
- [A Memory Augmented Architecture for Continuous Speaker Identification in Meetings](https://mlsanity.com/paper/2001.05118/) (2020)
- [Weakly-supervised Deep Cognate Detection Framework for Low-Resourced Languages Using Morphological Knowledge of Closely-Related Languages](https://mlsanity.com/paper/2311.05155/) (2023)

## Abstract (arXiv)

This paper provides a proof of concept that audio of tabletop role-playing games (TTRPG) could serve as a challenge for diarization systems. TTRPGs are carried out mostly by conversation. Participants often alter their voices to indicate that they are talking as a fictional character. Audio processing systems are susceptible to voice conversion with or without technological assistance. TTRPG present a conversational phenomenon in which voice conversion is an inherent characteristic for an immersive gaming experience. This could make it more challenging for diarizers to pick the real speaker and determine that impersonating is just that. We present the creation of a small TTRPG audio dataset and compare it against the AMI and the ICSI corpus. The performance of two diarizers, pyannote.audio and wespeaker, were evaluated. We observed that TTRPGs' properties result in a higher confusion rate for both diarizers. Additionally, wespeaker strongly underestimates the number of speakers in the TTRPG audio files. We propose TTRPG audio as a promising challenge for diarization systems.