# Exploring Speaker-Related Information in Spoken Language Understanding for Better Speaker Diarization

arXiv: https://arxiv.org/abs/2305.12927 · HTML: https://mlsanity.com/paper/2305.12927/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 34 references that are in the map.
- First posted on arXiv 22 May 2023.

## Builds on (most cited first, 22 of 22)

- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization](https://mlsanity.com/paper/2104.05938/) (2021)
- [A Review of Speaker Diarization: Recent Advances with Deep Learning](https://mlsanity.com/paper/2101.09624/) (2021)
- [End-to-End Neural Speaker Diarization with Self-attention](https://mlsanity.com/paper/1909.06247/) (2019)
- [Speaker Diarization with LSTM](https://mlsanity.com/paper/1710.10468/) (2017)
- [End-to-End Speaker Diarization for an Unknown Number of Speakers with Encoder-Decoder Based Attractors](https://mlsanity.com/paper/2005.09921/) (2020)
- [M2MeT: The ICASSP 2022 Multi-Channel Multi-Party Meeting Transcription Challenge](https://mlsanity.com/paper/2110.07393/) (2021)
- [AISHELL-4: An Open Source Dataset for Speech Enhancement, Separation, Recognition and Speaker Diarization in Conference Scenario](https://mlsanity.com/paper/2104.03603/) (2021)
- [ECAPA-TDNN Embeddings for Speaker Diarization](https://mlsanity.com/paper/2104.01466/) (2021)
- [Auto-Tuning Spectral Clustering for Speaker Diarization Using Normalized Maximum Eigengap](https://mlsanity.com/paper/2003.02405/) (2020)
- [Controllable Time-Delay Transformer for Real-Time Punctuation Prediction and Disfluency Detection](https://mlsanity.com/paper/2003.01309/) (2020)
- [Addressee and Response Selection in Multi-Party Conversations with Speaker Interaction RNNs](https://mlsanity.com/paper/1709.04005/) (2017)
- [Target-speaker Voice Activity Detection with Improved I-Vector Estimation for Unknown Number of Speaker](https://mlsanity.com/paper/2108.03342/) (2021)
- [BERTraffic: BERT-based Joint Speaker Role and Speaker Change Detection for Air Traffic Control Communications](https://mlsanity.com/paper/2110.05781/) (2021)
- [A Real-time Speaker Diarization System Based on Spatial Spectrum](https://mlsanity.com/paper/2107.09321/) (2021)
- [Universal ASR: Unifying Streaming and Non-Streaming ASR Using a Single Encoder-Decoder Model](https://mlsanity.com/paper/2010.14099/) (2020)
- [Speaker Overlap-aware Neural Diarization for Multi-party Meeting Analysis](https://mlsanity.com/paper/2211.10243/) (2022)
- [Speaker Role Contextual Modeling for Language Understanding and Dialogue Policy Learning](https://mlsanity.com/paper/1710.00164/) (2017)
- [Multimodal Clustering with Role Induced Constraints for Speaker Diarization](https://mlsanity.com/paper/2204.00657/) (2022)
- [Reformulating Speaker Diarization as Community Detection With Emphasis On Topological Structure](https://mlsanity.com/paper/2204.12112/) (2022)
- [Speaker Embedding-aware Neural Diarization: an Efficient Framework for Overlapping Speech Diarization in Meeting Scenarios](https://mlsanity.com/paper/2203.09767/) (2022)
- [PRISM: Pre-trained Indeterminate Speaker Representation Model for Speaker Diarization and Speaker Verification](https://mlsanity.com/paper/2205.07450/) (2022)

## Built on by (most cited first, 5 of 5)

- [Large Language Model Based Generative Error Correction: A Challenge and Baselines for Speech Recognition, Speaker Tagging, and Emotion Recognition](https://mlsanity.com/paper/2409.09785/) (2024)
- [Improving Speaker Diarization using Semantic Information: Joint Pairwise Constraints Propagation](https://mlsanity.com/paper/2309.10456/) (2023)
- [Do We Still Need Audio? Rethinking Speaker Diarization with a Text-Based Approach Using Multiple Prediction Models](https://mlsanity.com/paper/2506.11344/) (2025)
- [UniSLU: Unified Spoken Language Understanding from Heterogeneous Cross-Task Datasets](https://mlsanity.com/paper/2507.12951/) (2025)
- [CineSRD: Leveraging Visual, Acoustic, and Linguistic Cues for Open-World Visual Media Speaker Diarization](https://mlsanity.com/paper/2603.16966/) (2026)

## Abstract (arXiv)

Speaker diarization(SD) is a classic task in speech processing and is crucial in multi-party scenarios such as meetings and conversations. Current mainstream speaker diarization approaches consider acoustic information only, which result in performance degradation when encountering adverse acoustic conditions. In this paper, we propose methods to extract speaker-related information from semantic content in multi-party meetings, which, as we will show, can further benefit speaker diarization. We introduce two sub-tasks, Dialogue Detection and Speaker-Turn Detection, in which we effectively extract speaker information from conversational semantics. We also propose a simple yet effective algorithm to jointly model acoustic and semantic information and obtain speaker-identified texts. Experiments on both AISHELL-4 and AliMeeting datasets show that our method achieves consistent improvements over acoustic-only speaker diarization systems.