# Ask, Don't Guess: Targeted Clarification for Speech Recognition Errors

arXiv: https://arxiv.org/abs/2610.24809 · HTML: https://mlsanity.com/paper/2610.24809/

## Key facts

- No citing papers in the map yet (data through 19 Oct 2026).
- Placed in Speech recognition models and training, one of 11 directions in Speech and audio language models.
- Builds on 14 of its 37 references that are in the map.
- First posted on arXiv 17 Oct 2026.

## Placed in

- Speech and audio language models / [Speech recognition models and training](https://mlsanity.com/speech/speech-recognition-models-and-training/)

## Builds on (most cited first, 14 of 14)

- [MUSAN: A Music, Speech, and Noise Corpus](https://mlsanity.com/paper/1510.08484/) (2015)
- [CHiME-6 Challenge:Tackling Multispeaker Speech Recognition for Unsegmented Recordings](https://mlsanity.com/paper/2004.09249/) (2020)
- [Fast Conformer with Linearly Scalable Attention for Efficient Speech Recognition](https://mlsanity.com/paper/2305.05084/) (2023)
- [Bayesian HMM clustering of x-vector sequences (VBx) in speaker diarization: theory, implementation and analysis on standard tasks](https://mlsanity.com/paper/2012.14952/) (2020)
- [Generative Speech Recognition Error Correction with Large Language Models and Task-Activating Prompting](https://mlsanity.com/paper/2309.15649/) (2023)
- [Can Generative Large Language Models Perform ASR Error Correction?](https://mlsanity.com/paper/2307.04172/) (2023)
- [NOTSOFAR-1 Challenge: New Datasets, Baseline, and Tasks for Distant Meeting Transcription](https://mlsanity.com/paper/2401.08887/) (2024)
- [FastCorrect: Fast Error Correction with Edit Alignment for Automatic Speech Recognition](https://mlsanity.com/paper/2105.03842/) (2021)
- [DiPCo -- Dinner Party Corpus](https://mlsanity.com/paper/1909.13447/) (2019)
- [The CHiME-8 DASR Challenge for Generalizable and Array Agnostic Distant Automatic Speech Recognition and Diarization](https://mlsanity.com/paper/2407.16447/) (2024)
- [Confidence Estimation and Deletion Prediction Using Bidirectional Recurrent Neural Networks](https://mlsanity.com/paper/1810.13025/) (2018)
- [Multi-Task Learning for End-to-End ASR Word and Utterance Confidence with Deletion Prediction](https://mlsanity.com/paper/2104.12870/) (2021)
- [Word-level confidence estimation for RNN transducers](https://mlsanity.com/paper/2110.15222/) (2021)
- [Interactive In-Meeting Speaker Correction with Human Feedback](https://mlsanity.com/paper/2509.18377/) (2025)

## Abstract (arXiv)

Speech recognition in real time in far-field conversational settings still incurs significant errors. Identifying potential error spans during streaming ASR permits a recourse mechanism where the speaker can be asked to repeat the affected content under cleaner acoustic conditions. We propose PARDON, a new framework for online error localization, that identifies spans likely to benefit from this intervention. Our supervision uses clean audio to estimate how the recognizer's output might change under improved acoustic conditions. We probabilistically align noisy and clean decodes using a pair-HMM to obtain soft supervision. These soft targets train a counterfactual gain scorer by combining a suite of acoustic and language features to produce token-level scores, that are further grouped into spans for targeted recourse. An LLM uses the ASR transcript of the repetition to revise only the selected span. At a precision of 70%, PARDON achieves the highest recall by identifying up to 52% more erroneous tokens compared to the strongest baseline. Targeted repetition yields relative WER reductions of up to 24% under a budget that accounts for both interruption frequency and the lengths of re-uttered spans.