mlsanityData to 7 Oct 2026

A Comparative Study on Non-Autoregressive Modelings for Speech-to-Text Generation

arXiv 2110.05249 ↗First posted 11 Oct 2021eess.AS cs.CL cs.SDcited by 27

Not on the map. Why?

Abstract (arXiv)

Non-autoregressive (NAR) models simultaneously generate multiple outputs in a sequence, which significantly reduces the inference speed at the cost of accuracy drop compared to autoregressive baselines. Showing great potential for real-time applications, an increasing number of NAR models have been explored in different fields to mitigate the performance gap against AR models. In this work, we conduct a comparative study of various NAR modeling methods for end-to-end automatic speech recognition (ASR). Experiments are performed in the state-of-the-art setting using ESPnet. The results on various tasks provide interesting findings for developing an understanding of NAR ASR, such as the accuracy-speed trade-off and robustness against long-form utterances. We also show that the techniques can be combined for further improvement and applied to NAR end-to-end speech translation. All the implementations are publicly available to encourage further research in NAR speech processing.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 45 of its 66 references that are in the map. First posted on arXiv 11 Oct 2021.

Builds on

45 of 66 references.

References · 45

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  5. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  6. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  7. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  8. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  9. 2016Sequence-Level Knowledge Distillationcited by 761
  10. 2013Speech Recognition with Deep Recurrent Neural Networkscited by 666
  11. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 632
  12. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 554
33 more
  1. 2015Listen, Attend and Spellcited by 542
  2. 2017Non-Autoregressive Neural Machine Translationcited by 457
  3. 2015Attention-Based Models for Speech Recognitioncited by 400
  4. 2019A Comparative Study on Transformer vs RNN in Speech Applicationscited by 210
  5. 2019Levenshtein Transformercited by 197
  6. 2017State-of-the-art Speech Recognition With Sequence-to-Sequence Modelscited by 196
  7. 2018Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinementcited by 194
  8. 2020Recent Developments on ESPnet Toolkit Boosted by Conformercited by 132
  9. 2019Insertion Transformer: Flexible Sequence Generation via Insertion Operationscited by 123
  10. 2016Towards better decoding and language model integration in sequence to sequence modelscited by 100
  11. 2020Non-Autoregressive Machine Translation with Latent Alignmentscited by 96
  12. 2021Intermediate Loss Regularization for CTC-based Speech Recognitioncited by 95
  13. 2019QuartzNet: Deep Automatic Speech Recognition with 1D Time-Channel Separable Convolutionscited by 93
  14. 2019FlowSeq: Non-Autoregressive Conditional Sequence Generation with Generative Flowcited by 89
  15. 2019CIF: Continuous Integrate-and-Fire for End-to-End Speech Recognitioncited by 76
  16. 2020Mask CTC: Non-Autoregressive End-to-End ASR with CTC and Mask Predictcited by 75
  17. 2018End-to-End Non-Autoregressive Neural Machine Translation with Connectionist Temporal Classificationcited by 72
  18. 2020Imputer: Sequence Modelling via Imputation and Dynamic Programmingcited by 65
  19. 2019RWTH ASR Systems for LibriSpeech: Hybrid vs Attention -- w/o Data Augmentationcited by 63
  20. 2021Relaxing the Conditional Independence Assumption of CTC-based ASR by Conditioning on Intermediate Predictionscited by 61
  21. 2017Exploring Neural Transducers for End-to-End Speech Recognitioncited by 43
  22. 2020Align-Refine: Non-Autoregressive Speech Recognition via Iterative Realignmentcited by 37
  23. 2021Citrinet: Closing the Gap between Non-Autoregressive and Autoregressive End-to-End Models for Automatic Speech Recognitioncited by 33
  24. 2020Improved Mask-CTC for Non-Autoregressive End-to-End ASRcited by 32
  25. 2019Deja-vu: Double Feature Presentation and Iterated Loss in Deep Transformer Networkscited by 24
  26. 2020Spike-Triggered Non-Autoregressive Transformer for End-to-End Speech Recognitioncited by 24
  27. 2020Listen Attentively, and Spell Once: Whole Sentence Generation via a Non-Autoregressive Architecture for Low-Latency Speech Recognitioncited by 22
  28. 2019KERMIT: Generative Insertion-Based Modeling for Sequencescited by 21
  29. 2021Pushing the Limits of Non-Autoregressive Speech Recognitioncited by 18
  30. 2020Orthros: Non-autoregressive End-to-end Speech Translation with Dual-decodercited by 17
  31. 2020The 2020 ESPnet update: new features, broadened applications, performance improvements, and future planscited by 14
  32. 2020Insertion-Based Modeling for End-to-End Automatic Speech Recognitioncited by 13
  33. 2021Boundary and Context Aware Training for CIF-based Non-Autoregressive End-to-end ASRcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 10; 2023: 10; 2024: 3; 2025: 3; 2026: 1.

27 papers.

Sort:

By area:

  1. 2023End-to-End Speech Recognition: A Surveycited by 127
  2. 2022A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyondcited by 64
  3. 2024OWSM-CTC: An Open Encoder-Only Speech Foundation Model for Speech Recognition, Translation, and Language Identificationcited by 34
  4. 2023Improving Massively Multilingual ASR With Auxiliary CTC Objectivescited by 31
  5. 2022A context-aware knowledge transferring strategy for CTC-based ASRcited by 12
  6. 2022Blockwise Streaming Transformer for Spoken Language Understanding and Simultaneous Speech Translationcited by 11
  7. 2022BERT Meets CTC: New Formulation of End-to-End Speech Recognition with Pre-trained Masked Language Modelcited by 11
  8. 2023CTC-based Non-autoregressive Speech Translationcited by 9
  9. 2023CIF-T: A Novel CIF-based Transducer Architecture for Automatic Speech Recognitioncited by 6
  10. 2023A CTC Alignment-based Non-autoregressive Transformer for End-to-end Automatic Speech Recognitioncited by 5
  11. 2024InterBiasing: Boost Unseen Word Recognition through Biasing Intermediate Predictionscited by 5
  12. 2022Improving CTC-based ASR Models with Gated Interlayer Collaborationcited by 4
15 more
  1. 2022MLP-ASR: Sequence-length agnostic all-MLP architectures for speech recognitioncited by 3
  2. 2022Memory-Efficient Training of RNN-Transducer with Sampled Softmaxcited by 3
  3. 2023A Token-Wise Beam Search Algorithm for RNN-Tcited by 3
  4. 2023Neural Diarization with Non-autoregressive Intermediate Attractorscited by 3
  5. 2025WCTC-Biasing: Retraining-free Contextual Biasing ASR with Wildcard CTC-based Keyword Spotting and Inter-layer Biasingcited by 3
  6. 2023Unimodal Aggregation for CTC-based Speech Recognitioncited by 2
  7. 2022On the Prediction Network Architecture in RNN-T for ASRcited by 1
  8. 2023RenewNAT: Renewing Potential Translation for Non-Autoregressive Transformercited by 1
  9. 2024UniEnc-CASSNAT: An Encoder-only Non-autoregressive ASR for Speech SSL Modelscited by 1
  10. 2025DYNAC: Dynamic Vocabulary based Non-Autoregressive Contextualization for Speech Recognitioncited by 1
  11. 2022Alternate Intermediate Conditioning with Syllable-level and Character-level Targets for Japanese ASRcited by 0
  12. 2022InterMPL: Momentum Pseudo-Labeling with Intermediate CTC Losscited by 0
  13. 2023A Lexical-aware Non-autoregressive Transformer-based ASR Modelcited by 0
  14. 2025UMA-Split: unimodal aggregation for both English and Mandarin non-autoregressive speech recognitioncited by 0
  15. 2026Diffusion Language Models for Speech Recognitioncited by 0