mlsanityData to 7 Oct 2026

AG-LSEC: Audio Grounded Lexical Speaker Error Correction

arXiv 2406.17266 ↗First posted 25 Jun 2024eess.AS cs.AI cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

Speaker Diarization (SD) systems are typically audio-based and operate independently of the ASR system in traditional speech transcription pipelines and can have speaker errors due to SD and/or ASR reconciliation, especially around speaker turns and regions of speech overlap. To reduce these errors, a Lexical Speaker Error Correction (LSEC), in which an external language model provides lexical information to correct the speaker errors, was recently proposed. Though the approach achieves good Word Diarization error rate (WDER) improvements, it does not use any additional acoustic information and is prone to miscorrections. In this paper, we propose to enhance and acoustically ground the LSEC system with speaker scores directly derived from the existing SD pipeline. This approach achieves significant relative WDER reductions in the range of 25-40% over the audio-based SD, ASR system and beats the LSEC system by 15-25% relative on RT03-CTS, Callhome American English and Fisher datasets.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 31 of its 37 references that are in the map. First posted on arXiv 25 Jun 2024.

Builds on

31 of 37 references.

References · 31

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  3. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  4. 2017Generalized End-to-End Loss for Speaker Verificationcited by 290
  5. 2021A Review of Speaker Diarization: Recent Advances with Deep Learningcited by 122
  6. 2023Fast Conformer with Linearly Scalable Attention for Efficient Speech Recognitioncited by 120
  7. 2019End-to-End Neural Speaker Diarization with Permutation-Free Objectivescited by 86
  8. 2019Transformer-based Acoustic Modeling for Hybrid Speech Recognitioncited by 86
  9. 2019End-to-End Neural Speaker Diarization with Self-attentioncited by 74
  10. 2017Speaker Diarization with LSTMcited by 73
  11. 2019Joint Speech Recognition and Speaker Diarization via Sequence Transductioncited by 49
  12. 2021ECAPA-TDNN Embeddings for Speaker Diarizationcited by 47
19 more
  1. 2020End-to-End Multi-speaker Speech Recognition with Transformercited by 42
  2. 2022Streaming Multi-Talker ASR with Token-Level Serialized Output Trainingcited by 37
  3. 2020Continuous Speech Separation with Conformercited by 36
  4. 2020Integrating end-to-end neural and clustering-based diarization: Getting the best of both worldscited by 33
  5. 2021Turn-to-Diarize: Online Speaker Diarization Constrained by Transformer Transducer Speaker Turn Detectioncited by 31
  6. 2023MossFormer: Pushing the Performance Limit of Monaural Speech Separation using Gated Single-Head Transformer with Convolution-Augmented Joint Self-Attentionscited by 27
  7. 2020Streaming Multi-speaker ASR with RNN-Tcited by 22
  8. 2020The RWTH ASR System for TED-LIUM Release 2: Improving Hybrid HMM with SpecAugmentcited by 19
  9. 2024The Sound of Healthcare: Improving Medical Transcription ASR Accuracy with Large Language Modelscited by 17
  10. 2024DiarizationLM: Speaker Diarization Post-Processing with Large Language Modelscited by 16
  11. 2023Enhancing Speaker Diarization with Large Language Models: A Contextual Beam Search Approachcited by 15
  12. 2020Speaker Diarization with Lexical Informationcited by 14
  13. 2023Lexical Speaker Error Correction: Leveraging Language Models for Speaker Diarization Error Correctioncited by 14
  14. 2022Highly Efficient Real-Time Streaming and Fully On-Device Speaker Diarization with Multi-Stage Clusteringcited by 10
  15. 2019End-to-end training of time domain audio separation and recognitioncited by 8
  16. 2022A Conformer Based Acoustic Model for Robust Automatic Speech Recognitioncited by 6
  17. 2023CASA-ASR: Context-Aware Speaker-Attributed ASRcited by 6
  18. 2021Directed Speech Separation for Automatic Speech Recognition of Long Form Conversational Speechcited by 5
  19. 2023Mixture Encoder for Joint Speech Separation and Recognitioncited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025SEAL: Speaker Error Correction using Acoustic-conditioned Large Language Modelscited by 1
  2. 2026Iterative LLM-based improvement for French Clinical Interview Transcription and Speaker Diarizationcited by 0
  3. 2026Mitigating Speaker Leakage in Cascaded Multi-talker ASR with Diarization-based Transcript Correctioncited by 0