mlsanityData to 14 Oct 2026

Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognition

arXiv 2206.08317 ↗First posted 16 Jun 2022cs.SD cs.CL eess.AScited by 144

Not on the map. Why?

Abstract (arXiv)

Transformers have recently dominated the ASR field. Although able to yield good performance, they involve an autoregressive (AR) decoder to generate tokens one by one, which is computationally inefficient. To speed up inference, non-autoregressive (NAR) methods, e.g. single-step NAR, were designed, to enable parallel generation. However, due to an independence assumption within the output tokens, performance of single-step NAR is inferior to that of AR models, especially with a large-scale corpus. There are two challenges to improving single-step NAR: Firstly to accurately predict the number of output tokens and extract hidden variables; secondly, to enhance modeling of interdependence between output tokens. To tackle both challenges, we propose a fast and accurate parallel transformer, termed Paraformer. This utilizes a continuous integrate-and-fire based predictor to predict the number of tokens and generate hidden variables. A glancing language model (GLM) sampler then generates semantic embeddings to enhance the NAR decoder's ability to model context interdependence. Finally, we design a strategy to generate negative samples for minimum word error rate training to further improve performance. Experiments using the public AISHELL-1, AISHELL-2 benchmark, and an industrial-level 20,000 hour task demonstrate that the proposed Paraformer can attain comparable performance to the state-of-the-art AR transformer, with more than 10x speedup.

Key facts. Cited by 144 papers in the map (data through 14 Oct 2026). Builds on 27 of its 32 references that are in the map. First posted on arXiv 16 Jun 2022.

Builds on

27 of 32 references.

References · 27

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  3. 2013Speech Recognition with Deep Recurrent Neural Networkscited by 669
  4. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 558
  5. 2015Listen, Attend and Spellcited by 543
  6. 2017Non-Autoregressive Neural Machine Translationcited by 463
  7. 2017AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baselinecited by 373
  8. 2017OpenNMT: Open-source Toolkit for Neural Machine Translationcited by 251
  9. 2018Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinementcited by 195
  10. 2018AISHELL-2: Transforming Mandarin ASR Research Into Industrial Scalecited by 128
  11. 2020Glancing Transformer for Non-Autoregressive Neural Machine Translationcited by 103
  12. 2019CIF: Continuous Integrate-and-Fire for End-to-End Speech Recognitioncited by 77
15 more
  1. 2020Mask CTC: Non-Autoregressive End-to-End ASR with CTC and Mask Predictcited by 76
  2. 2017Minimum Word Error Rate Training for Attention-based Sequence-to-Sequence Modelscited by 68
  3. 2020Improved Mask-CTC for Non-Autoregressive End-to-End ASRcited by 32
  4. 2021Fast End-to-End Speech Recognition via Non-Autoregressive Models and Cross-Modal Knowledge Transferring from BERTcited by 31
  5. 2020Spike-Triggered Non-Autoregressive Transformer for End-to-End Speech Recognitioncited by 24
  6. 2020CASS-NAT: CTC Alignment-based Single Step Non-autoregressive Transformer for Speech Recognitioncited by 24
  7. 2020Non-Autoregressive Transformer ASR with CTC-Enhanced Decoder Inputcited by 24
  8. 2022Improving non-autoregressive end-to-end speech recognition with pre-trained acoustic and language modelscited by 21
  9. 2020SAN-M: Memory Equipped Self-Attention for End-to-End Speech Recognitioncited by 16
  10. 2022Improving CTC-based speech recognition via knowledge transferring from pre-trained language modelscited by 14
  11. 2020Streaming Chunk-Aware Multihead Attention for Online End-to-End Speech Recognitioncited by 12
  12. 2021Streaming End-to-End ASR based on Blockwise Non-Autoregressive Modelscited by 11
  13. 2020Universal ASR: Unifying Streaming and Non-Streaming ASR Using a Single Encoder-Decoder Modelcited by 9
  14. 2021An Improved Single Step Non-autoregressive Transformer for Automatic Speech Recognitioncited by 8
  15. 2021Non-autoregressive Transformer with Unified Bidirectional Decoder for Automatic Speech Recognitioncited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 15; 2024: 30; 2025: 61; 2026: 38.

144 papers.

Sort:

By area:

  1. 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Modelscited by 288
  2. 2025Kimi-Audio Technical Reportcited by 222
  3. 2024Freeze-Omni: A Smart and Low Latency Speech-to-speech Dialogue Model with Frozen LLMcited by 141
  4. 2024FunAudioLLM: Voice Understanding and Generation Foundation Models for Natural Interaction Between Humans and LLMscited by 122
  5. 2023FunASR: A Fundamental End-to-End Speech Recognition Toolkitcited by 95
  6. 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 85
  7. 2025Step-Audio: Unified Understanding and Generation in Intelligent Speech Interactioncited by 83
  8. 2023LauraGPT: Listen, Attend, Understand, and Regenerate Audio with GPTcited by 78
  9. 2026MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interactioncited by 78
  10. 2025Llasa: Scaling Train-Time and Inference-Time Compute for Llama-based Speech Synthesiscited by 70
  11. 2025MiMo-Audio: Audio Language Models are Few-Shot Learnerscited by 65
  12. 2024Enhance Reasoning for Large Language Models in the Game Werewolfcited by 46
38 more
  1. 2024Baichuan-Omni Technical Reportcited by 42
  2. 2023Transformers in Speech Processing: A Surveycited by 39
  3. 2024Recurrent Drafter for Fast Speculative Decoding in Large Language Modelscited by 37
  4. 2025VITA-Audio: Fast Interleaved Cross-Modal Token Generation for Efficient Large Speech-Language Modelcited by 31
  5. 2025ZipVoice: Fast and High-Quality Zero-Shot Text-to-Speech with Flow Matchingcited by 30
  6. 2025VibeVoice Technical Reportcited by 29
  7. 2025IndexTTS: An Industrial-Level Controllable and Efficient Zero-Shot Text-To-Speech Systemcited by 28
  8. 2026OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Modelscited by 25
  9. 2025M2-omni: Advancing Omni-MLLM for Comprehensive Modality Support with Competitive Performancecited by 19
  10. 2023SeACo-Paraformer: A Non-Autoregressive ASR System with Flexible and Effective Hotword Customization Abilitycited by 18
  11. 2025MoonCast: High-Quality Zero-Shot Podcast Generationcited by 18
  12. 2025X-Streamer: Unified Human World Modeling with Audiovisual Interactioncited by 18
  13. 2024Comparing Discrete and Continuous Space LLMs for Speech Recognitioncited by 15
  14. 2025Differentiable Reward Optimization for LLM based TTS systemcited by 15
  15. 2025NVSpeech: An Integrated and Scalable Pipeline for Human-Like Speech Modeling with Paralinguistic Vocalizationscited by 15
  16. 2025SpeechJudge: Towards Human-Level Judgment for Speech Naturalnesscited by 15
  17. 2024Unveiling the Potential of LLM-Based ASR on Chinese Open-Source Datasetscited by 13
  18. 2025Easy Turn: Integrating Acoustic and Linguistic Modalities for Robust Turn-Taking in Full-Duplex Spoken Dialogue Systemscited by 13
  19. 2023Knowledge Transfer from Pre-trained Language Models to Cif-based Speech Recognizers via Hierarchical Distillationcited by 11
  20. 2025TaoAvatar: Real-Time Lifelike Full-Body Talking Avatars for Augmented Reality via 3D Gaussian Splattingcited by 11
  21. 2023Achieving Timestamp Prediction While Recognizing with Non-Autoregressive End-to-End ASR Modelcited by 10
  22. 2024TouchTTS: An Embarrassingly Simple TTS Framework that Everyone Can Touchcited by 10
  23. 2026PASK: Toward Intent-Aware Proactive Agents with Long-Term Memorycited by 9
  24. 2025Dolphin: A Large-Scale Automatic Speech Recognition Model for Eastern Languagescited by 8
  25. 2025ChatAnyone: Stylized Real-time Portrait Video Generation with Hierarchical Motion Diffusion Modelcited by 8
  26. 2025WenetSpeech-Chuan: A Large-Scale Sichuanese Corpus with Rich Annotation for Dialectal Speech Processingcited by 8
  27. 2024MDPE: A Multimodal Deception Dataset with Personality and Emotional Characteristicscited by 7
  28. 2025Low-Resource Domain Adaptation for Speech LLMs via Text-Only Fine-Tuningcited by 7
  29. 2025Step-Audio-AQAA: a Fully End-to-End Expressive Large Audio Language Modelcited by 7
  30. 2025ZipVoice-Dialog: Non-Autoregressive Spoken Dialogue Generation with Flow Matchingcited by 7
  31. 2025UniSS: Unified Expressive Speech-to-Speech Translation with Your Voicecited by 7
  32. 2023CIF-T: A Novel CIF-based Transducer Architecture for Automatic Speech Recognitioncited by 6
  33. 2025SynParaSpeech: Automated Synthesis of Paralinguistic Datasets for Speech Generation and Understandingcited by 6
  34. 2025MNV-17: A High-Quality Performative Mandarin Dataset for Nonverbal Vocalization Recognition in Speechcited by 6
  35. 2025M3-TTS: Multi-modal DiT Alignment & Mel-latent for Zero-shot High-fidelity Speech Synthesiscited by 6
  36. 2023Semi-Autoregressive Streaming ASR With Label Contextcited by 5
  37. 2024CNVSRC 2023: The First Chinese Continuous Visual Speech Recognition Challengecited by 5
  38. 2024Imperceptible Rhythm Backdoor Attacks: Exploring Rhythm Transformation for Embedding Undetectable Vulnerabilities on Speech Recognitioncited by 5

50 most cited of 144 All 144 (JSON)