mlsanityData to 7 Oct 2026

Proactive for Uncertainty: Cause-Aware Error Diagnosis and Interactive Clarification for Spoken Dialogue Systems

arXiv 2605.25404 ↗First posted 25 May 2026cs.CL eess.AScited by 0

Not on the map. Why?

Abstract (arXiv)

Cascaded Automatic Speech Recognition - Large Language Model (ASR-LLM) pipelines remain popular for industrial Spoken Dialogue Systems (SDS), primarily because their decoupled design ensures perceptual verifiability. However, cascaded systems suffer from error propagation, as transcription failures inevitably cascade to subsequent components, thereby degrading the final interaction quality. Although ASR confidence scores offer a simple filter for unreliable inputs, this approach is fundamentally limited because it typically fails to detect deletion errors or to distinguish between acoustic (inability to hear clearly) and linguistic (inability to understand) mismatches, both of which require targeted recovery strategies. In this paper, we propose a cause-aware error recovery paradigm that fundamentally rethinks robustness in SDS. Unlike traditional confidence filtering, we introduce a suite of small precision-focused detectors that exploit deep ASR latent representations to disentangle token-level errors into perception, comprehension, and deletion failures. This fine-grained diagnostic intelligence empowers the LLM to orchestrate targeted, multi-turn clarification strategies, effectively transforming ambiguous signals into seamless user interactions. Experimental results validate the precision of our approach, which more than doubles the recall on domain-shift errors (57.96% vs. 23.66%) compared to baselines. Crucially, this diagnostic precision yields up to a 31% reduction in WER and a 19% improvement on the downstream task across diverse accents, distortions, and domains.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 26 of its 50 references that are in the map. First posted on arXiv 25 May 2026.

Builds on

26 of 50 references.

References · 26

  1. 2021AST: Audio Spectrogram Transformercited by 634
  2. 2024Qwen2-Audio Technical Reportcited by 543
  3. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 536
  4. 2015MUSAN: A Music, Speech, and Noise Corpuscited by 400
  5. 2021GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audiocited by 391
  6. 2025CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-trainingcited by 142
  7. 2019Asking Clarifying Questions in Open-Domain Information-Seeking Conversationscited by 136
  8. 2024AudioBench: A Universal Benchmark for Audio Large Language Modelscited by 124
  9. 2024FunAudioLLM: Voice Understanding and Generation Foundation Models for Natural Interaction Between Humans and LLMscited by 118
  10. 2025Step-Audio 2 Technical Reportcited by 101
  11. 2025Step-Audio: Unified Understanding and Generation in Intelligent Speech Interactioncited by 83
  12. 2024Language Model Can Listen While Speakingcited by 62
14 more
  1. 2017Convolutional Recurrent Neural Networks for Polyphonic Sound Event Detectioncited by 46
  2. 2020ClarQ: A large-scale and diverse dataset for Clarification Question Generationcited by 31
  3. 2021The Accented English Speech Recognition Challenge 2020: Open Datasets, Tracks, Baselines, Results and Methodscited by 24
  4. 2021Learning Word-Level Confidence For Subword End-to-End ASRcited by 22
  5. 2021An evaluation of word-level confidence estimation for end-to-end automatic speech recognitioncited by 18
  6. 2020Resolving Intent Ambiguities by Retrieving Discriminative Clarifying Questionscited by 16
  7. 2025X-Talk: On the Underestimated Potential of Modular Speech-to-Speech Dialogue Systemcited by 9
  8. 2024Towards ASR Robust Spoken Language Understanding Through In-Context Learning With Word Confusion Networkscited by 8
  9. 2025Evaluating ASR Confidence Scores for Automated Error Detection in User-Assisted Correction Interfacescited by 6
  10. 2025Contextual ASR Error Handling with LLMs Augmentation for Goal-Oriented Conversational AIcited by 5
  11. 2025SPGISpeech 2.0: Transcribed multi-speaker financial audio for speaker-tagged transcriptioncited by 5
  12. 2021Word-level confidence estimation for RNN transducerscited by 3
  13. 2023BLSTM-Based Confidence Estimation for End-to-End Speech Recognitioncited by 2
  14. 2024HAINAN: Fast and Accurate Transducer for Hybrid-Autoregressive ASRcited by 2

Built on by

Not cited yet.