# Speaker Diarization with Lexical Information

arXiv: https://arxiv.org/abs/2004.06756 · HTML: https://mlsanity.com/paper/2004.06756/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Apr 2020.

## Built on by (most cited first, 14 of 14)

- [A Review of Speaker Diarization: Recent Advances with Deep Learning](https://mlsanity.com/paper/2101.09624/) (2021)
- [Transcribe-to-Diarize: Neural Speaker Diarization for Unlimited Number of Speakers using End-to-End Speaker-Attributed ASR](https://mlsanity.com/paper/2110.03151/) (2021)
- [DiarizationLM: Speaker Diarization Post-Processing with Large Language Models](https://mlsanity.com/paper/2401.03506/) (2024)
- [Lexical Speaker Error Correction: Leveraging Language Models for Speaker Diarization Error Correction](https://mlsanity.com/paper/2306.09313/) (2023)
- [Large Language Model Based Generative Error Correction: A Challenge and Baselines for Speech Recognition, Speaker Tagging, and Emotion Recognition](https://mlsanity.com/paper/2409.09785/) (2024)
- [DiariST: Streaming Speech Translation with Speaker Diarization](https://mlsanity.com/paper/2309.08007/) (2023)
- [ASR-Aware End-to-end Neural Diarization](https://mlsanity.com/paper/2202.01286/) (2022)
- [The Conversational Short-phrase Speaker Diarization (CSSD) Task: Dataset, Evaluation Metric and Baselines](https://mlsanity.com/paper/2208.08042/) (2022)
- [AG-LSEC: Audio Grounded Lexical Speaker Error Correction](https://mlsanity.com/paper/2406.17266/) (2024)
- [Encoder-decoder multimodal speaker change detection](https://mlsanity.com/paper/2306.00680/) (2023)
- [Speaker Diarization of Scripted Audiovisual Content](https://mlsanity.com/paper/2308.02160/) (2023)
- [Generation of Speaker Representations Using Heterogeneous Training Batch Assembly](https://mlsanity.com/paper/2203.16646/) (2022)
- [Playing with Voices: Tabletop Role-Playing Game Recordings as a Diarization Challenge](https://mlsanity.com/paper/2502.12714/) (2025)
- [Training-Free Affinity Fusion of Neural and Embedding-Based Speaker Diarization](https://mlsanity.com/paper/2609.39162/) (2026)

## Abstract (arXiv)

This work presents a novel approach for speaker diarization to leverage lexical information provided by automatic speech recognition. We propose a speaker diarization system that can incorporate word-level speaker turn probabilities with speaker embeddings into a speaker clustering process to improve the overall diarization accuracy. To integrate lexical and acoustic information in a comprehensive way during clustering, we introduce an adjacency matrix integration for spectral clustering. Since words and word boundary information for word-level speaker turn probability estimation are provided by a speech recognition system, our proposed method works without any human intervention for manual transcriptions. We show that the proposed method improves diarization performance on various evaluation datasets compared to the baseline diarization system using acoustic information only in speaker embeddings.