# Improved Mask-CTC for Non-Autoregressive End-to-End ASR

arXiv: https://arxiv.org/abs/2010.13270 · HTML: https://mlsanity.com/paper/2010.13270/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Oct 2020.

## Built on by (most cited first, 30 of 32)

- [Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognition](https://mlsanity.com/paper/2206.08317/) (2022)
- [Recent Advances in End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/2111.01690/) (2021)
- [A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond](https://mlsanity.com/paper/2204.09269/) (2022)
- [Relaxing the Conditional Independence Assumption of CTC-based ASR by Conditioning on Intermediate Predictions](https://mlsanity.com/paper/2104.02724/) (2021)
- [A Comparative Study on Non-Autoregressive Modelings for Speech-to-Text Generation](https://mlsanity.com/paper/2110.05249/) (2021)
- [Pushing the Limits of Non-Autoregressive Speech Recognition](https://mlsanity.com/paper/2104.03416/) (2021)
- [Deliberation of Streaming RNN-Transducer by Non-autoregressive Decoding](https://mlsanity.com/paper/2112.11442/) (2021)
- [Streaming End-to-End ASR based on Blockwise Non-Autoregressive Models](https://mlsanity.com/paper/2107.09428/) (2021)
- [BERT Meets CTC: New Formulation of End-to-End Speech Recognition with Pre-trained Masked Language Model](https://mlsanity.com/paper/2210.16663/) (2022)
- [Non-autoregressive End-to-end Speech Translation with Parallel Autoregressive Rescoring](https://mlsanity.com/paper/2109.04411/) (2021)
- [Fast-MD: Fast Multi-Decoder End-to-End Speech Translation with Non-Autoregressive Hidden Intermediates](https://mlsanity.com/paper/2109.12804/) (2021)
- [CTC-based Non-autoregressive Speech Translation](https://mlsanity.com/paper/2305.17358/) (2023)
- [Non-autoregressive Mandarin-English Code-switching Speech Recognition](https://mlsanity.com/paper/2104.02258/) (2021)
- [An Improved Single Step Non-autoregressive Transformer for Automatic Speech Recognition](https://mlsanity.com/paper/2106.09885/) (2021)
- [BECTRA: Transducer-based End-to-End ASR with BERT-Enhanced Encoder](https://mlsanity.com/paper/2211.00792/) (2022)
- [TSNAT: Two-Step Non-Autoregressvie Transformer Models for Speech Recognition](https://mlsanity.com/paper/2104.01522/) (2021)
- [Cross-Modal ASR Post-Processing System for Error Correction and Utterance Rejection](https://mlsanity.com/paper/2201.03313/) (2022)
- [Non-autoregressive Error Correction for CTC-based ASR with Phone-conditioned Masked LM](https://mlsanity.com/paper/2209.04062/) (2022)
- [InterAug: Augmenting Noisy Intermediate Predictions for CTC-based ASR](https://mlsanity.com/paper/2204.00174/) (2022)
- [Improving CTC-based ASR Models with Gated Interlayer Collaboration](https://mlsanity.com/paper/2205.12462/) (2022)
- [Dynamic Alignment Mask CTC: Improved Mask-CTC with Aligned Cross Entropy](https://mlsanity.com/paper/2303.07687/) (2023)
- [Tailored Design of Audio-Visual Speech Recognition Models using Branchformers](https://mlsanity.com/paper/2407.06606/) (2024)
- [HAINAN: Fast and Accurate Transducer for Hybrid-Autoregressive ASR](https://mlsanity.com/paper/2410.02597/) (2024)
- [Streaming Align-Refine for Non-autoregressive Deliberation](https://mlsanity.com/paper/2204.07556/) (2022)
- [Towards Effective and Efficient Non-autoregressive Decoding Using Block-based Attention Mask](https://mlsanity.com/paper/2406.10034/) (2024)
- [Audio-Conditioned Diffusion LLMs for ASR and Deliberation Processing](https://mlsanity.com/paper/2509.16622/) (2025)
- [An Investigation of Enhancing CTC Model for Triggered Attention-based Streaming ASR](https://mlsanity.com/paper/2110.10402/) (2021)
- [Linguistic-Enhanced Transformer with CTC Embedding for Speech Recognition](https://mlsanity.com/paper/2210.14725/) (2022)
- [A Lexical-aware Non-autoregressive Transformer-based ASR Model](https://mlsanity.com/paper/2305.10839/) (2023)
- [Unsupervised ASR via Cross-Lingual Pseudo-Labeling](https://mlsanity.com/paper/2305.13330/) (2023)

## Abstract (arXiv)

For real-world deployment of automatic speech recognition (ASR), the system is desired to be capable of fast inference while relieving the requirement of computational resources. The recently proposed end-to-end ASR system based on mask-predict with connectionist temporal classification (CTC), Mask-CTC, fulfills this demand by generating tokens in a non-autoregressive fashion. While Mask-CTC achieves remarkably fast inference speed, its recognition performance falls behind that of conventional autoregressive (AR) systems. To boost the performance of Mask-CTC, we first propose to enhance the encoder network architecture by employing a recently proposed architecture called Conformer. Next, we propose new training and decoding methods by introducing auxiliary objective to predict the length of a partial target sequence, which allows the model to delete or insert tokens during inference. Experimental results on different ASR tasks show that the proposed approaches improve Mask-CTC significantly, outperforming a standard CTC model (15.5% rightarrow 9.1% WER on WSJ). Moreover, Mask-CTC now achieves competitive results to AR models with no degradation of inference speed (< 0.1 RTF using CPU). We also show a potential application of Mask-CTC to end-to-end speech translation.