mlsanityData to 7 Oct 2026

Lexical Speaker Error Correction: Leveraging Language Models for Speaker Diarization Error Correction

arXiv 2306.09313 ↗First posted 15 Jun 2023eess.AS cs.AI cs.CLcited by 14

Not on the map. Why?

Abstract (arXiv)

Speaker diarization (SD) is typically used with an automatic speech recognition (ASR) system to ascribe speaker labels to recognized words. The conventional approach reconciles outputs from independently optimized ASR and SD systems, where the SD system typically uses only acoustic information to identify the speakers in the audio stream. This approach can lead to speaker errors especially around speaker turns and regions of speaker overlap. In this paper, we propose a novel second-pass speaker error correction system using lexical information, leveraging the power of modern language models (LMs). Our experiments across multiple telephony datasets show that our approach is both effective and robust. Training and tuning only on the Fisher dataset, this error correction approach leads to relative word-level diarization error rate (WDER) reductions of 15-30% on three telephony datasets: RT03-CTS, Callhome American English and held-out portions of Fisher.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 22 of its 34 references that are in the map. First posted on arXiv 15 Jun 2023.

Builds on

22 of 34 references.

References · 22

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  5. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  6. 2020In defence of metric learning for speaker recognitioncited by 123
  7. 2021A Review of Speaker Diarization: Recent Advances with Deep Learningcited by 122
  8. 2019Transformer-based Acoustic Modeling for Hybrid Speech Recognitioncited by 86
  9. 2017Speaker Diarization with LSTMcited by 73
  10. 2023PLACES: Prompting Language Models for Social Conversation Synthesiscited by 72
  11. 2017Deep Speaker: an End-to-End Neural Speaker Embedding Systemcited by 56
  12. 2019Joint Speech Recognition and Speaker Diarization via Sequence Transductioncited by 49
10 more
  1. 2021ECAPA-TDNN Embeddings for Speaker Diarizationcited by 47
  2. 2018Fully Supervised Speaker Diarizationcited by 46
  3. 2020MatchboxNet: 1D Time-Channel Separable Convolutional Neural Network Architecture for Speech Commands Recognitioncited by 36
  4. 2021Turn-to-Diarize: Online Speaker Diarization Constrained by Transformer Transducer Speaker Turn Detectioncited by 31
  5. 2020The RWTH ASR System for TED-LIUM Release 2: Improving Hybrid HMM with SpecAugmentcited by 19
  6. 2020Speaker Diarization with Lexical Informationcited by 14
  7. 2018Multimodal Speaker Segmentation and Diarization using Lexical and Acoustic Cues via Sequence to Sequence Neural Networkscited by 12
  8. 2022Highly Efficient Real-Time Streaming and Fully On-Device Speaker Diarization with Multi-Stage Clusteringcited by 10
  9. 2022A Conformer Based Acoustic Model for Robust Automatic Speech Recognitioncited by 6
  10. 2017Speaker Diarization using Deep Recurrent Convolutional Neural Networks for Speaker Embeddingscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 7; 2025: 2; 2026: 2.

14 papers.

Sort:

  1. 2024SpeechVerse: A Large-scale Generalizable Audio Language Modelcited by 62
  2. 2024DiarizationLM: Speaker Diarization Post-Processing with Large Language Modelscited by 16
  3. 2024Large Language Model Based Generative Error Correction: A Challenge and Baselines for Speech Recognition, Speaker Tagging, and Emotion Recognitioncited by 10
  4. 2024LLM-based speaker diarization correction: A generalizable approachcited by 8
  5. 2023DiaCorrect: Error Correction Back-end For Speaker Diarizationcited by 3
  6. 2023Improving Speaker Diarization using Semantic Information: Joint Pairwise Constraints Propagationcited by 3
  7. 2024AG-LSEC: Audio Grounded Lexical Speaker Error Correctioncited by 3
  8. 2024Integrating Audio, Visual, and Semantic Information for Enhanced Multimodal Speaker Diarizationcited by 2
  9. 2023Speaker Diarization of Scripted Audiovisual Contentcited by 1
  10. 2025SEAL: Speaker Error Correction using Acoustic-conditioned Large Language Modelscited by 1
  11. 2025Do We Still Need Audio? Rethinking Speaker Diarization with a Text-Based Approach Using Multiple Prediction Modelscited by 1
  12. 2024Speaker Tagging Correction With Non-Autoregressive Language Modelscited by 0
2 more
  1. 2026Iterative LLM-based improvement for French Clinical Interview Transcription and Speaker Diarizationcited by 0
  2. 2026Mitigating Speaker Leakage in Cascaded Multi-talker ASR with Diarization-based Transcript Correctioncited by 0