# ASR Error Detection via Audio-Transcript entailment

arXiv: https://arxiv.org/abs/2207.10849 · HTML: https://mlsanity.com/paper/2207.10849/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 32 references that are in the map.
- First posted on arXiv 22 Jul 2022.

## Builds on (most cited first, 12 of 12)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT](https://mlsanity.com/paper/2004.14786/) (2020)
- [Generating SOAP Notes from Doctor-Patient Conversations Using Modular Summarization Techniques](https://mlsanity.com/paper/2005.01795/) (2020)
- [ASR Error Correction and Domain Adaptation Using Machine Translation](https://mlsanity.com/paper/2003.07692/) (2020)
- [What does BERT know about books, movies and music? Probing BERT for Conversational Recommendation](https://mlsanity.com/paper/2007.15356/) (2020)
- [Towards an Automated SOAP Note: Classifying Utterances from Medical Conversations](https://mlsanity.com/paper/2007.08749/) (2020)
- [Black-box Adaptation of ASR for Accented Speech](https://mlsanity.com/paper/2006.13519/) (2020)
- [RED-ACE: Robust Error Detection for ASR using Confidence Embeddings](https://mlsanity.com/paper/2203.07172/) (2022)
- [Investigating the Effects of Word Substitution Errors on Sentence Embeddings](https://mlsanity.com/paper/1811.07021/) (2018)
- [Weakly Supervised Medication Regimen Extraction from Medical Conversations](https://mlsanity.com/paper/2010.05317/) (2020)

## Built on by (most cited first, 1 of 1)

- [Leveraging Fine-grained Error Correction in Korean Speech Recognition for Consultation Services](https://mlsanity.com/paper/2609.09889/) (2026)

## Abstract (arXiv)

Despite improved performances of the latest Automatic Speech Recognition (ASR) systems, transcription errors are still unavoidable. These errors can have a considerable impact in critical domains such as healthcare, when used to help with clinical documentation. Therefore, detecting ASR errors is a critical first step in preventing further error propagation to downstream applications. To this end, we propose a novel end-to-end approach for ASR error detection using audio-transcript entailment. To the best of our knowledge, we are the first to frame this problem as an end-to-end entailment task between the audio segment and its corresponding transcript segment. Our intuition is that there should be a bidirectional entailment between audio and transcript when there is no recognition error and vice versa. The proposed model utilizes an acoustic encoder and a linguistic encoder to model the speech and transcript respectively. The encoded representations of both modalities are fused to predict the entailment. Since doctor-patient conversations are used in our experiments, a particular emphasis is placed on medical terms. Our proposed model achieves classification error rates (CER) of 26.2% on all transcription errors and 23% on medical errors specifically, leading to improvements upon a strong baseline by 12% and 15.4%, respectively.