mlsanityData to 7 Oct 2026

Non-Autoregressive Transformer ASR with CTC-Enhanced Decoder Input

arXiv 2010.15025 ↗First posted 28 Oct 2020cs.SD cs.CL eess.AScited by 24

Not on the map. Why?

Abstract (arXiv)

Non-autoregressive (NAR) transformer models have achieved significantly inference speedup but at the cost of inferior accuracy compared to autoregressive (AR) models in automatic speech recognition (ASR). Most of the NAR transformers take a fixed-length sequence filled with MASK tokens or a redundant sequence copied from encoder states as decoder input, they cannot provide efficient target-side information thus leading to accuracy degradation. To address this problem, we propose a CTC-enhanced NAR transformer, which generates target sequence by refining predictions of the CTC module. Experimental results show that our method outperforms all previous NAR counterparts and achieves 50x faster decoding speed than a strong AR baseline with only 0.0 ~ 0.3 absolute CER degradation on Aishell-1 and Aishell-2 datasets.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 6; 2023: 3; 2024: 3; 2025: 2; 2026: 1.

24 papers.

Sort:

  1. 2022Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognitioncited by 139
  2. 2021Recent Advances in End-to-End Automatic Speech Recognitioncited by 138
  3. 2022A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyondcited by 64
  4. 2021Wav-BERT: Cooperative Acoustic and Linguistic Representation Learning for Low-Resource Speech Recognitioncited by 16
  5. 2021Non-autoregressive Transformer-based End-to-end ASR using BERTcited by 13
  6. 2022A context-aware knowledge transferring strategy for CTC-based ASRcited by 12
  7. 2021Fast-MD: Fast Multi-Decoder End-to-End Speech Translation with Non-Autoregressive Hidden Intermediatescited by 9
  8. 2021An Improved Single Step Non-autoregressive Transformer for Automatic Speech Recognitioncited by 8
  9. 2021Consistent Training and Decoding For End-to-end Speech Recognition Using Lattice-free MMIcited by 8
  10. 2024Automatic Speech Recognition with BERT and CTC Transformers: A Reviewcited by 8
  11. 2021TSNAT: Two-Step Non-Autoregressvie Transformer Models for Speech Recognitioncited by 7
  12. 20224D ASR: Joint modeling of CTC, Attention, Transducer, and Mask-Predict decoderscited by 7
12 more
  1. 2021Decoupling recognition and transcription in Mandarin ASRcited by 6
  2. 2021Non-autoregressive Transformer with Unified Bidirectional Decoder for Automatic Speech Recognitioncited by 5
  3. 2023Fingerspelling PoseNet: Enhancing Fingerspelling Translation with Pose-Based Transformer Modelscited by 5
  4. 2022Sub-8-Bit Quantization Aware Training for 8-Bit Neural Network Accelerator with On-Device Speech Recognitioncited by 4
  5. 2024Joint Beam Search Integrating CTC, Attention, and Transducer Decoderscited by 4
  6. 2022Adaptive Sparse and Monotonic Attention for Transformer-based Automatic Speech Recognitioncited by 2
  7. 2023Dynamic Alignment Mask CTC: Improved Mask-CTC with Aligned Cross Entropycited by 2
  8. 2024Towards Effective and Efficient Non-autoregressive Decoding Using Block-based Attention Maskcited by 1
  9. 2025Enhancing the Robustness of Contextual ASR to Varying Biasing Information Volumes Through Purified Semantic Correlation Joint Modelingcited by 1
  10. 2025Audio-Conditioned Diffusion LLMs for ASR and Deliberation Processingcited by 1
  11. 2026Pay Attention to CTC: Fast and Robust Pseudo-Labelling for Unified Speech Recognitioncited by 1
  12. 2023A Lexical-aware Non-autoregressive Transformer-based ASR Modelcited by 0