mlsanityData to 14 Oct 2026

FunASR: A Fundamental End-to-End Speech Recognition Toolkit

arXiv 2305.11013 ↗First posted 18 May 2023cs.SD cs.CL eess.AScited by 95

Not on the map. Why?

Abstract (arXiv)

This paper introduces FunASR, an open-source speech recognition toolkit designed to bridge the gap between academic research and industrial applications. FunASR offers models trained on large-scale industrial corpora and the ability to deploy them in applications. The toolkit's flagship model, Paraformer, is a non-autoregressive end-to-end speech recognition model that has been trained on a manually annotated Mandarin speech recognition dataset that contains 60,000 hours of speech. To improve the performance of Paraformer, we have added timestamp prediction and hotword customization capabilities to the standard Paraformer backbone. In addition, to facilitate model deployment, we have open-sourced a voice activity detection model based on the Feedforward Sequential Memory Network (FSMN-VAD) and a text post-processing punctuation model based on the controllable time-delay Transformer (CT-Transformer), both of which were trained on industrial corpora. These functional modules provide a solid foundation for building high-precision long audio speech recognition services. Compared to other models trained on open datasets, Paraformer demonstrates superior performance.

Key facts. Cited by 95 papers in the map (data through 14 Oct 2026). Builds on 27 of its 31 references that are in the map. First posted on arXiv 18 May 2023.

Builds on

27 of 31 references.

References · 27

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  3. 2013Speech Recognition with Deep Recurrent Neural Networkscited by 669
  4. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 558
  5. 2015Listen, Attend and Spellcited by 543
  6. 2017AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baselinecited by 373
  7. 2021WenetSpeech: A 10000+ Hours Multi-domain Mandarin Corpus for Speech Recognitioncited by 197
  8. 2022Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognitioncited by 144
  9. 2018Deep context: end-to-end contextual speech recognitioncited by 137
  10. 2018AISHELL-2: Transforming Mandarin ASR Research Into Industrial Scalecited by 128
  11. 2021WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkitcited by 125
  12. 2019CIF: Continuous Integrate-and-Fire for End-to-End Speech Recognitioncited by 77
15 more
  1. 2022WeNet 2.0: More Productive End-to-End Speech Recognition Toolkitcited by 47
  2. 2018Deep-FSMN for Large Vocabulary Continuous Speech Recognitioncited by 28
  3. 2020Controllable Time-Delay Transformer for Real-Time Punctuation Prediction and Disfluency Detectioncited by 23
  4. 2020SAN-M: Memory Equipped Self-Attention for End-to-End Speech Recognitioncited by 16
  5. 2020Streaming Chunk-Aware Multihead Attention for Online End-to-End Speech Recognitioncited by 12
  6. 2022Fast and parallel decoding for transducercited by 12
  7. 2021DISC: A Dynamic Shape Compiler for Machine Learning Workloadscited by 10
  8. 2023Achieving Timestamp Prediction While Recognizing with Non-Autoregressive End-to-End ASR Modelcited by 10
  9. 2020Universal ASR: Unifying Streaming and Non-Streaming ASR Using a Single Encoder-Decoder Modelcited by 9
  10. 2022PaddleSpeech: An Easy-to-Use All-in-One Speech Toolkitcited by 9
  11. 2022ConvRNN-T: Convolutional Augmented Recurrent Neural Network Transducers for Streaming Speech Recognitioncited by 8
  12. 2021Extremely Low Footprint End-to-End ASR System for Smart Devicecited by 7
  13. 2022Memory-Efficient Training of RNN-Transducer with Sampled Softmaxcited by 3
  14. 2022Separator-Transducer-Segmenter: Streaming Recognition and Segmentation of Multi-party Speechcited by 3
  15. 2022Multiple-hypothesis RNN-T Loss for Unsupervised Fine-tuning and Self-training of Neural Transducercited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 21; 2025: 35; 2026: 34.

95 papers.

Sort:

By area:

  1. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
  2. 2024Qwen2-Audio Technical Reportcited by 560
  3. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 361
  4. 2025CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-trainingcited by 150
  5. 2024FunAudioLLM: Voice Understanding and Generation Foundation Models for Natural Interaction Between Humans and LLMscited by 122
  6. 2025Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokenscited by 113
  7. 2025IndexTTS2: A Breakthrough in Emotionally Expressive and Duration-Controlled Auto-Regressive Zero-Shot Text-to-Speechcited by 90
  8. 2023LauraGPT: Listen, Attend, Understand, and Regenerate Audio with GPTcited by 78
  9. 2025Llasa: Scaling Train-Time and Inference-Time Compute for Llama-based Speech Synthesiscited by 70
  10. 2024OmniFlatten: An End-to-end GPT Model for Seamless Voice Conversationcited by 60
  11. 2025Uni-Sign: Toward Unified Sign Language Understanding at Scalecited by 56
  12. 2024InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactionscited by 48
38 more
  1. 2025SongGen: A Single Stage Auto-regressive Transformer for Text-to-Song Generationcited by 29
  2. 2025SplitMeanFlow: Interval Splitting Consistency in Few-Step Generative Modelingcited by 26
  3. 2026JavisDiT++: Unified Modeling and Optimization for Joint Audio-Video Generationcited by 25
  4. 2025F5R-TTS: Improving Flow-Matching based Text-to-Speech with Group Relative Policy Optimizationcited by 24
  5. 2024Speech Translation with Speech Foundation Models and Large Language Models: What is There and What is Missing?cited by 23
  6. 2023SeACo-Paraformer: A Non-Autoregressive ASR System with Flexible and Effective Hotword Customization Abilitycited by 18
  7. 2025ContextASR-Bench: A Massive Contextual Speech Recognition Benchmarkcited by 15
  8. 2025NVSpeech: An Integrated and Scalable Pipeline for Human-Like Speech Modeling with Paralinguistic Vocalizationscited by 15
  9. 2025SpeechJudge: Towards Human-Level Judgment for Speech Naturalnesscited by 15
  10. 2026JoyAI-VL-Interaction: Real-Time Vision-Language Interaction Intelligencecited by 14
  11. 2025TaDiCodec: Text-aware Diffusion Speech Tokenizer for Speech Language Modelingcited by 13
  12. 2026SocialOmni: Benchmarking Audio-Visual Social Interactivity in Omni Modelscited by 13
  13. 2025AudioGen-Omni: A Unified Multimodal Diffusion Transformer for Video-Synchronized Audio, Speech, and Song Generationcited by 12
  14. 2025FLM-Audio: Natural Monologues Improves Native Full-Duplex Chatbots via Dual Trainingcited by 11
  15. 2025LiveTalk: Real-Time Multimodal Interactive Video Diffusion via Improved On-Policy Distillationcited by 9
  16. 2026Fish Audio S2 Technical Reportcited by 9
  17. 2024An Effective Mixture-Of-Experts Approach For Code-Switching Speech Recognition Leveraging Encoder Disentanglementcited by 8
  18. 2025DMOSpeech 2: Reinforcement Learning for Duration Prediction in Metric-Optimized Speech Synthesiscited by 8
  19. 2025MGM-Omni: Scaling Omni LLMs to Personalized Long-Horizon Speechcited by 7
  20. 2026IndexTTS 2.5 Technical Reportcited by 6
  21. 2024YingSound: Video-Guided Sound Effects Generation with Multi-modal Chain-of-Thought Controlscited by 5
  22. 2025Leveraging LLM and Self-Supervised Training Models for Speech Recognition in Chinese Dialects: A Comparative Analysiscited by 5
  23. 2025Vevo2: A Unified and Controllable Framework for Speech and Singing Voice Generationcited by 5
  24. 2026Mega-ASR: Towards In-the-wild^2 Speech Recognition via Scaling up Real-world Acoustic Simulationcited by 5
  25. 2024Spontaneous Speech-Based Suicide Risk Detection Using Whisper and Large Language Modelscited by 4
  26. 2024MM-KWS: Multi-modal Prompts for Multilingual User-defined Keyword Spottingcited by 4
  27. 2025Pose as a Modality: A Psychology-Inspired Network for Personality Recognition with a New Multimodal Datasetcited by 4
  28. 2023Improving Speaker Diarization using Semantic Information: Joint Pairwise Constraints Propagationcited by 3
  29. 2024Bailing-TTS: Chinese Dialectal Speech Synthesis Towards Human-like Spontaneous Representationcited by 3
  30. 2024Large Generative Model-assisted Talking-face Semantic Communication Systemcited by 3
  31. 2025EmoHead: Emotional Talking Head via Manipulating Semantic Expression Parameterscited by 3
  32. 2025VietASR: Achieving Industry-level Vietnamese ASR with 50-hour labeled data and Large-Scale Speech Pretrainingcited by 3
  33. 2025Inference-time Scaling for Diffusion-based Audio Super-resolutioncited by 3
  34. 2025VoiceCraft-X: Unifying Multilingual, Voice-Cloning Speech Synthesis and Speech Editingcited by 3
  35. 2026FunCineForge: A Unified Dataset Toolkit and Model for Zero-Shot Movie Dubbing in Diverse Cinematic Scenescited by 3
  36. 2026LongCat-AudioDiT: High-Fidelity Diffusion Text-to-Speech in the Waveform Latent Spacecited by 3
  37. 2024CoCoGesture: Toward Coherent Co-speech 3D Gesture Generation in the Wildcited by 2
  38. 2024Qifusion-Net: Layer-adapted Stream/Non-stream Model for End-to-End Multi-Accent Speech Recognitioncited by 2

50 most cited of 95 All 95 (JSON)