# FastCorrect 2: Fast Error Correction on Multiple Candidates for Automatic Speech Recognition

arXiv: https://arxiv.org/abs/2109.14420 · HTML: https://mlsanity.com/paper/2109.14420/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 26 references that are in the map.
- First posted on arXiv 29 Sep 2021.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baseline](https://mlsanity.com/paper/1709.05522/) (2017)
- [Levenshtein Transformer](https://mlsanity.com/paper/1905.11006/) (2019)
- [Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinement](https://mlsanity.com/paper/1802.06901/) (2018)
- [ASR Error Correction and Domain Adaptation Using Machine Translation](https://mlsanity.com/paper/2003.07692/) (2020)
- [FastCorrect: Fast Error Correction with Edit Alignment for Automatic Speech Recognition](https://mlsanity.com/paper/2105.03842/) (2021)
- [Non-Autoregressive Neural Machine Translation with Enhanced Decoder Input](https://mlsanity.com/paper/1812.09664/) (2018)
- [Fine-Tuning by Curriculum Learning for Non-Autoregressive Neural Machine Translation](https://mlsanity.com/paper/1911.08717/) (2019)
- [Improving Readability for Automatic Speech Recognition Transcription](https://mlsanity.com/paper/2004.04438/) (2020)
- [Highway Long Short-Term Memory RNNs for Distant Speech Recognition](https://mlsanity.com/paper/1510.08983/) (2015)
- [Learning from Past Mistakes: Improving Automatic Speech Recognition Output via Noisy-Clean Phrase Context Modeling](https://mlsanity.com/paper/1802.02607/) (2018)
- [Multi-Encoder Learning and Stream Fusion for Transformer-Based End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/2104.00120/) (2021)

## Built on by (most cited first, 27 of 27)

- [Can Generative Large Language Models Perform ASR Error Correction?](https://mlsanity.com/paper/2307.04172/) (2023)
- [N-best T5: Robust ASR Error Correction using Multiple Input Hypotheses and Constrained Decoding Space](https://mlsanity.com/paper/2303.00456/) (2023)
- [HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Models](https://mlsanity.com/paper/2309.15701/) (2023)
- [Large Language Models are Efficient Learners of Noise-Robust Speech Recognition](https://mlsanity.com/paper/2401.10446/) (2024)
- [ASR Error Correction using Large Language Models](https://mlsanity.com/paper/2409.09554/) (2024)
- [Non-autoregressive Generative Models for Reranking Recommendation](https://mlsanity.com/paper/2402.06871/) (2024)
- [Listen Again and Choose the Right Answer: A New Paradigm for Automatic Speech Recognition with Large Language Models](https://mlsanity.com/paper/2405.10025/) (2024)
- [Mask the Correct Tokens: An Embarrassingly Simple Approach for Error Correction](https://mlsanity.com/paper/2211.13252/) (2022)
- [SoftCorrect: Error Correction with Soft Detection for Automatic Speech Recognition](https://mlsanity.com/paper/2212.01039/) (2022)
- [RED-ACE: Robust Error Detection for ASR using Confidence Embeddings](https://mlsanity.com/paper/2203.07172/) (2022)
- [Transcormer: Transformer for Sentence Scoring with Sliding Language Modeling](https://mlsanity.com/paper/2205.12986/) (2022)
- [Measuring the Effect of Transcription Noise on Downstream Language Understanding Tasks](https://mlsanity.com/paper/2502.13645/) (2025)
- [Fewer Hallucinations, More Verification: A Three-Stage LLM-Based Framework for ASR Error Correction](https://mlsanity.com/paper/2505.24347/) (2025)
- [Revisiting ASR Error Correction with Specialized Models](https://mlsanity.com/paper/2405.15216/) (2024)
- [DANCER: Entity Description Augmented Named Entity Corrector for Automatic Speech Recognition](https://mlsanity.com/paper/2403.17645/) (2024)
- [Can Visual Context Improve Automatic Speech Recognition for an Embodied Agent?](https://mlsanity.com/paper/2210.13189/) (2022)
- [Full-text Error Correction for Chinese Speech Recognition with Large Language Model](https://mlsanity.com/paper/2409.07790/) (2024)
- [Large Language Model Should Understand Pinyin for Chinese ASR Error Correction](https://mlsanity.com/paper/2409.13262/) (2024)
- [Enhancing conversational quality in language learning chatbots: An evaluation of GPT4 for ASR error correction](https://mlsanity.com/paper/2307.09744/) (2023)
- [Error Correction by Paying Attention to Both Acoustic and Confidence References for Automatic Speech Recognition](https://mlsanity.com/paper/2407.12817/) (2024)
- [Listening and Seeing Again: Generative Error Correction for Audio-Visual Speech Recognition](https://mlsanity.com/paper/2501.04038/) (2025)
- [HypR: A comprehensive study for ASR hypothesis revising with a reference corpus](https://mlsanity.com/paper/2309.09838/) (2023)
- [PERL: Pinyin Enhanced Rephrasing Language Model for Chinese ASR N-best Error Correction](https://mlsanity.com/paper/2412.03230/) (2024)
- [Chain of Correction for Full-text Speech Recognition with Large Language Models](https://mlsanity.com/paper/2504.01519/) (2025)
- [Optimized Tokenization for Transcribed Error Correction](https://mlsanity.com/paper/2310.10704/) (2023)
- [Pinyin Regularization in Error Correction for Chinese Speech Recognition with Large Language Models](https://mlsanity.com/paper/2407.01909/) (2024)
- [Grammatical Error Correction Evaluation by Optimally Transporting Edit Representation](https://mlsanity.com/paper/2602.05419/) (2026)

## Abstract (arXiv)

Error correction is widely used in automatic speech recognition (ASR) to post-process the generated sentence, and can further reduce the word error rate (WER). Although multiple candidates are generated by an ASR system through beam search, current error correction approaches can only correct one sentence at a time, failing to leverage the voting effect from multiple candidates to better detect and correct error tokens. In this work, we propose FastCorrect 2, an error correction model that takes multiple ASR candidates as input for better correction accuracy. FastCorrect 2 adopts non-autoregressive generation for fast inference, which consists of an encoder that processes multiple source sentences and a decoder that generates the target sentence in parallel from the adjusted source sentence, where the adjustment is based on the predicted duration of each source token. However, there are some issues when handling multiple source sentences. First, it is non-trivial to leverage the voting effect from multiple source sentences since they usually vary in length. Thus, we propose a novel alignment algorithm to maximize the degree of token alignment among multiple sentences in terms of token and pronunciation similarity. Second, the decoder can only take one adjusted source sentence as input, while there are multiple source sentences. Thus, we develop a candidate predictor to detect the most suitable candidate for the decoder. Experiments on our inhouse dataset and AISHELL-1 show that FastCorrect 2 can further reduce the WER over the previous correction model with single candidate by 3.2% and 2.6%, demonstrating the effectiveness of leveraging multiple candidates in ASR error correction. FastCorrect 2 achieves better performance than the cascaded re-scoring and correction pipeline and can serve as a unified post-processing module for ASR.