mlsanityData to 14 Oct 2026

Align-Refine: Non-Autoregressive Speech Recognition via Iterative Realignment

arXiv 2010.14233 ↗First posted 24 Oct 2020eess.AS cs.CL cs.LGcited by 37

Not on the map. Why?

Abstract (arXiv)

Non-autoregressive models greatly improve decoding speed over typical sequence-to-sequence models, but suffer from degraded performance. Infilling and iterative refinement models make up some of this gap by editing the outputs of a non-autoregressive model, but are constrained in the edits that they can make. We propose iterative realignment, where refinements occur over latent alignments rather than output sequence space. We demonstrate this in speech recognition with Align-Refine, an end-to-end Transformer-based model which refines connectionist temporal classification (CTC) alignments to allow length-changing insertions and deletions. Align-Refine outperforms Imputer and Mask-CTC, matching an autoregressive baseline on WSJ at 1/14th the real-time factor and attaining a LibriSpeech test-other WER of 9.0% without an LM. Our model is strong even in one iteration with a shallower decoder.

Key facts. Cited by 37 papers in the map (data through 14 Oct 2026). First posted on arXiv 24 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 12; 2022: 9; 2023: 5; 2024: 6; 2025: 5.

37 papers.

Sort:

By area:

  1. 2021Intermediate Loss Regularization for CTC-based Speech Recognitioncited by 96
  2. 2022A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyondcited by 64
  3. 2021Relaxing the Conditional Independence Assumption of CTC-based ASR by Conditioning on Intermediate Predictionscited by 61
  4. 2021PIMNet: A Parallel, Iterative and Mimicking Network for Scene Text Recognitioncited by 41
  5. 2024OWSM-CTC: An Open Encoder-Only Speech Foundation Model for Speech Recognition, Translation, and Language Identificationcited by 37
  6. 2025A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Modelscited by 30
  7. 2021A Comparative Study on Non-Autoregressive Modelings for Speech-to-Text Generationcited by 28
  8. 2022Recent Advances in Neural Text Generation: A Task-Agnostic Surveycited by 14
  9. 2021Layer Pruning on Demand with Intermediate CTCcited by 13
  10. 2021Hierarchical Conditional End-to-End ASR with CTC and Multi-Granular Subword Unitscited by 13
  11. 2021Deliberation of Streaming RNN-Transducer by Non-autoregressive Decodingcited by 13
  12. 2022A Character-Level Length-Control Algorithm for Non-Autoregressive Sentence Summarizationcited by 13
25 more
  1. 2023IPAD: Iterative, Parallel, and Diffusion-based Network for Scene Text Recognitioncited by 12
  2. 2021Streaming End-to-End ASR based on Blockwise Non-Autoregressive Modelscited by 11
  3. 2022BERT Meets CTC: New Formulation of End-to-End Speech Recognition with Pre-trained Masked Language Modelcited by 11
  4. 2021Non-autoregressive Mandarin-English Code-switching Speech Recognitioncited by 8
  5. 2021An Improved Single Step Non-autoregressive Transformer for Automatic Speech Recognitioncited by 8
  6. 2022Streaming parallel transducer beam search with fast-slow cascaded encoderscited by 8
  7. 2022InterAug: Augmenting Noisy Intermediate Predictions for CTC-based ASRcited by 5
  8. 2022Better Intermediates Improve CTC Inferencecited by 5
  9. 2023A CTC Alignment-based Non-autoregressive Transformer for End-to-end Automatic Speech Recognitioncited by 5
  10. 2024Exploring and Improving Drafts in Blockwise Parallel Decodingcited by 5
  11. 2024InterBiasing: Boost Unseen Word Recognition through Biasing Intermediate Predictionscited by 5
  12. 2023Lego-Features: Exporting modular encoder features for streaming and deliberation ASRcited by 3
  13. 2021Speaker conditioning of acoustic models using affine transformation for multi-speaker speech recognitioncited by 2
  14. 2023Masked Audio Text Encoders are Effective Multi-Modal Rescorerscited by 2
  15. 2025DYNAC: Dynamic Vocabulary based Non-Autoregressive Contextualization for Speech Recognitioncited by 2
  16. 2022Streaming Align-Refine for Non-autoregressive Deliberationcited by 1
  17. 2024UniEnc-CASSNAT: An Encoder-only Non-autoregressive ASR for Speech SSL Modelscited by 1
  18. 2024Towards Effective and Efficient Non-autoregressive Decoding Using Block-based Attention Maskcited by 1
  19. 2025A Non-autoregressive Model for Joint STT and TTScited by 1
  20. 2025Audio-Conditioned Diffusion LLMs for ASR and Deliberation Processingcited by 1
  21. 2021Exploring Non-Autoregressive End-To-End Neural Modeling For English Mispronunciation Detection And Diagnosiscited by 0
  22. 2022Alternate Intermediate Conditioning with Syllable-level and Character-level Targets for Japanese ASRcited by 0
  23. 2023Leveraging Language ID to Calculate Intermediate CTC Loss for Enhanced Code-Switching Speech Recognitioncited by 0
  24. 2024Enhancing Code-Switching ASR Leveraging Non-Peaky CTC Loss and Deep Language Posterior Injectioncited by 0
  25. 2025M-CIF: Multi-Scale Alignment For CIF-Based Non-Autoregressive ASRcited by 0