# SoftCorrect: Error Correction with Soft Detection for Automatic Speech Recognition

arXiv: https://arxiv.org/abs/2212.01039 · HTML: https://mlsanity.com/paper/2212.01039/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 51 references that are in the map.
- First posted on arXiv 2 Dec 2022.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [Non-Autoregressive Neural Machine Translation](https://mlsanity.com/paper/1711.02281/) (2017)
- [AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baseline](https://mlsanity.com/paper/1709.05522/) (2017)
- [WenetSpeech: A 10000+ Hours Multi-domain Mandarin Corpus for Speech Recognition](https://mlsanity.com/paper/2110.03370/) (2021)
- [GECToR -- Grammatical Error Correction: Tag, Not Rewrite](https://mlsanity.com/paper/2005.12592/) (2020)
- [Fully Non-autoregressive Neural Machine Translation: Tricks of the Trade](https://mlsanity.com/paper/2012.15833/) (2020)
- [Non-Autoregressive Machine Translation with Latent Alignments](https://mlsanity.com/paper/2004.07437/) (2020)
- [End-to-End Non-Autoregressive Neural Machine Translation with Connectionist Temporal Classification](https://mlsanity.com/paper/1811.04719/) (2018)
- [Deliberation Model Based Two-Pass End-to-End Speech Recognition](https://mlsanity.com/paper/2003.07962/) (2020)
- [ASR Error Correction and Domain Adaptation Using Machine Translation](https://mlsanity.com/paper/2003.07692/) (2020)
- [FastCorrect: Fast Error Correction with Edit Alignment for Automatic Speech Recognition](https://mlsanity.com/paper/2105.03842/) (2021)
- [Citrinet: Closing the Gap between Non-Autoregressive and Autoregressive End-to-End Models for Automatic Speech Recognition](https://mlsanity.com/paper/2104.01721/) (2021)
- [Joint Contextual Modeling for ASR Correction and Language Understanding](https://mlsanity.com/paper/2002.00750/) (2020)
- [Transformer Based Deliberation for Two-Pass Speech Recognition](https://mlsanity.com/paper/2101.11577/) (2021)
- [FastCorrect 2: Fast Error Correction on Multiple Candidates for Automatic Speech Recognition](https://mlsanity.com/paper/2109.14420/) (2021)
- [Improving Readability for Automatic Speech Recognition Transcription](https://mlsanity.com/paper/2004.04438/) (2020)
- [An Empirical Study of Efficient ASR Rescoring with Transformers](https://mlsanity.com/paper/1910.11450/) (2019)
- [Learning from Past Mistakes: Improving Automatic Speech Recognition Output via Noisy-Clean Phrase Context Modeling](https://mlsanity.com/paper/1802.02607/) (2018)
- [Mask the Correct Tokens: An Embarrassingly Simple Approach for Error Correction](https://mlsanity.com/paper/2211.13252/) (2022)
- [Have best of both worlds: two-pass hybrid and E2E cascading framework for speech recognition](https://mlsanity.com/paper/2110.04891/) (2021)
- [Cross-Modal ASR Post-Processing System for Error Correction and Utterance Rejection](https://mlsanity.com/paper/2201.03313/) (2022)

## Built on by (most cited first, 9 of 9)

- [HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Models](https://mlsanity.com/paper/2309.15701/) (2023)
- [It's Never Too Late: Fusing Acoustic Information into Large Language Models for Automatic Speech Recognition](https://mlsanity.com/paper/2402.05457/) (2024)
- [Fewer Hallucinations, More Verification: A Three-Stage LLM-Based Framework for ASR Error Correction](https://mlsanity.com/paper/2505.24347/) (2025)
- [Enhancing conversational quality in language learning chatbots: An evaluation of GPT4 for ASR error correction](https://mlsanity.com/paper/2307.09744/) (2023)
- [Error Correction by Paying Attention to Both Acoustic and Confidence References for Automatic Speech Recognition](https://mlsanity.com/paper/2407.12817/) (2024)
- [HypR: A comprehensive study for ASR hypothesis revising with a reference corpus](https://mlsanity.com/paper/2309.09838/) (2023)
- [Enhancing AAC Software for Dysarthric Speakers in e-Health Settings: An Evaluation Using TORGO](https://mlsanity.com/paper/2411.00980/) (2024)
- [Optimized Tokenization for Transcribed Error Correction](https://mlsanity.com/paper/2310.10704/) (2023)
- [Leveraging Fine-grained Error Correction in Korean Speech Recognition for Consultation Services](https://mlsanity.com/paper/2609.09889/) (2026)

## Abstract (arXiv)

Error correction in automatic speech recognition (ASR) aims to correct those incorrect words in sentences generated by ASR models. Since recent ASR models usually have low word error rate (WER), to avoid affecting originally correct tokens, error correction models should only modify incorrect words, and therefore detecting incorrect words is important for error correction. Previous works on error correction either implicitly detect error words through target-source attention or CTC (connectionist temporal classification) loss, or explicitly locate specific deletion/substitution/insertion errors. However, implicit error detection does not provide clear signal about which tokens are incorrect and explicit error detection suffers from low detection accuracy. In this paper, we propose SoftCorrect with a soft error detection mechanism to avoid the limitations of both explicit and implicit error detection. Specifically, we first detect whether a token is correct or not through a probability produced by a dedicatedly designed language model, and then design a constrained CTC loss that only duplicates the detected incorrect tokens to let the decoder focus on the correction of error tokens. Compared with implicit error detection with CTC loss, SoftCorrect provides explicit signal about which words are incorrect and thus does not need to duplicate every token but only incorrect tokens; compared with explicit error detection, SoftCorrect does not detect specific deletion/substitution/insertion errors but just leaves it to CTC loss. Experiments on AISHELL-1 and Aidatatang datasets show that SoftCorrect achieves 26.1% and 9.4% CER reduction respectively, outperforming previous works by a large margin, while still enjoying fast speed of parallel generation.