# Do We Still Need Audio? Rethinking Speaker Diarization with a Text-Based Approach Using Multiple Prediction Models

arXiv: https://arxiv.org/abs/2506.11344 · HTML: https://mlsanity.com/paper/2506.11344/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 31 references that are in the map.
- First posted on arXiv 12 Jun 2025.

## Builds on (most cited first, 13 of 13)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [Powerset multi-class cross entropy loss for neural speaker diarization](https://mlsanity.com/paper/2310.13025/) (2023)
- [The fifth 'CHiME' Speech Separation and Recognition Challenge: Dataset, task and baselines](https://mlsanity.com/paper/1803.10609/) (2018)
- [DailyTalk: Spoken Dialogue Dataset for Conversational Text-to-Speech](https://mlsanity.com/paper/2207.01063/) (2022)
- [TitaNet: Neural Model for speaker representation with 1D Depth-wise separable convolutions and global context](https://mlsanity.com/paper/2110.04410/) (2021)
- [Joint Speech Recognition and Speaker Diarization via Sequence Transduction](https://mlsanity.com/paper/1907.05337/) (2019)
- [Transcribe-to-Diarize: Neural Speaker Diarization for Unlimited Number of Speakers using End-to-End Speaker-Attributed ASR](https://mlsanity.com/paper/2110.03151/) (2021)
- [Enhancing Speaker Diarization with Large Language Models: A Contextual Beam Search Approach](https://mlsanity.com/paper/2309.05248/) (2023)
- [Lexical Speaker Error Correction: Leveraging Language Models for Speaker Diarization Error Correction](https://mlsanity.com/paper/2306.09313/) (2023)
- [Exploring Speaker-Related Information in Spoken Language Understanding for Better Speaker Diarization](https://mlsanity.com/paper/2305.12927/) (2023)
- [TOLD: A Novel Two-Stage Overlap-Aware Framework for Speaker Diarization](https://mlsanity.com/paper/2303.05397/) (2023)

## Built on by (most cited first, 1 of 1)

- [ASR-Synchronized Speaker-Role Diarization](https://mlsanity.com/paper/2507.17765/) (2025)

## Abstract (arXiv)

We present a novel approach to Speaker Diarization (SD) by leveraging text-based methods focused on Sentence-level Speaker Change Detection within dialogues. Unlike audio-based SD systems, which are often challenged by audio quality and speaker similarity, our approach utilizes the dialogue transcript alone. Two models are developed: the Single Prediction Model (SPM) and the Multiple Prediction Model (MPM), both of which demonstrate significant improvements in identifying speaker changes, particularly in short conversations. Our findings, based on a curated dataset encompassing diverse conversational scenarios, reveal that the text-based SD approach, especially the MPM, performs competitively against state-of-the-art audio-based SD systems, with superior performance in short conversational contexts. This paper not only showcases the potential of leveraging linguistic features for SD but also highlights the importance of integrating semantic understanding into SD systems, opening avenues for future research in multimodal and semantic feature-based diarization.