# RED-ACE: Robust Error Detection for ASR using Confidence Embeddings

arXiv: https://arxiv.org/abs/2203.07172 · HTML: https://mlsanity.com/paper/2203.07172/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 24 references that are in the map.
- First posted on arXiv 14 Mar 2022.

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [ASR Error Correction and Domain Adaptation Using Machine Translation](https://mlsanity.com/paper/2003.07692/) (2020)
- [FastCorrect: Fast Error Correction with Edit Alignment for Automatic Speech Recognition](https://mlsanity.com/paper/2105.03842/) (2021)
- [Confidence Estimation for Attention-based Sequence-to-sequence Models for Speech Recognition](https://mlsanity.com/paper/2010.11428/) (2020)
- [FastCorrect 2: Fast Error Correction on Multiple Candidates for Automatic Speech Recognition](https://mlsanity.com/paper/2109.14420/) (2021)
- [Improving Readability for Automatic Speech Recognition Transcription](https://mlsanity.com/paper/2004.04438/) (2020)
- [Learning Word-Level Confidence For Subword End-to-End ASR](https://mlsanity.com/paper/2103.06716/) (2021)
- [Black-box Adaptation of ASR for Accented Speech](https://mlsanity.com/paper/2006.13519/) (2020)

## Built on by (most cited first, 8 of 8)

- [LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations](https://mlsanity.com/paper/2410.02707/) (2024)
- [Multi-stage Large Language Model Correction for Speech Recognition](https://mlsanity.com/paper/2310.11532/) (2023)
- [Adapting an Unadaptable ASR System](https://mlsanity.com/paper/2306.01208/) (2023)
- [VoiceSHIELD-Small: Real-Time Malicious Speech Detection and Transcription](https://mlsanity.com/paper/2603.07708/) (2026)
- [ASR Error Detection via Audio-Transcript entailment](https://mlsanity.com/paper/2207.10849/) (2022)
- [Two Heads Are Better Than One: Audio-Visual Speech Error Correction with Dual Hypotheses](https://mlsanity.com/paper/2510.13281/) (2025)
- [Optimized Tokenization for Transcribed Error Correction](https://mlsanity.com/paper/2310.10704/) (2023)
- [Leveraging Fine-grained Error Correction in Korean Speech Recognition for Consultation Services](https://mlsanity.com/paper/2609.09889/) (2026)

## Abstract (arXiv)

ASR Error Detection (AED) models aim to post-process the output of Automatic Speech Recognition (ASR) systems, in order to detect transcription errors. Modern approaches usually use text-based input, comprised solely of the ASR transcription hypothesis, disregarding additional signals from the ASR model. Instead, we propose to utilize the ASR system's word-level confidence scores for improving AED performance. Specifically, we add an ASR Confidence Embedding (ACE) layer to the AED model's encoder, allowing us to jointly encode the confidence scores and the transcribed text into a contextualized representation. Our experiments show the benefits of ASR confidence scores for AED, their complementary effect over the textual signal, as well as the effectiveness and robustness of ACE for combining these signals. To foster further research, we publish a novel AED dataset consisting of ASR outputs on the LibriSpeech corpus with annotated transcription errors.