# FunASR: A Fundamental End-to-End Speech Recognition Toolkit

arXiv: https://arxiv.org/abs/2305.11013 · HTML: https://mlsanity.com/paper/2305.11013/

## Key facts

- Cited by 95 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 31 references that are in the map.
- First posted on arXiv 18 May 2023.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Speech Recognition with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1303.5778/) (2013)
- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [Listen, Attend and Spell](https://mlsanity.com/paper/1508.01211/) (2015)
- [AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baseline](https://mlsanity.com/paper/1709.05522/) (2017)
- [WenetSpeech: A 10000+ Hours Multi-domain Mandarin Corpus for Speech Recognition](https://mlsanity.com/paper/2110.03370/) (2021)
- [Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognition](https://mlsanity.com/paper/2206.08317/) (2022)
- [Deep context: end-to-end contextual speech recognition](https://mlsanity.com/paper/1808.02480/) (2018)
- [AISHELL-2: Transforming Mandarin ASR Research Into Industrial Scale](https://mlsanity.com/paper/1808.10583/) (2018)
- [WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkit](https://mlsanity.com/paper/2102.01547/) (2021)
- [CIF: Continuous Integrate-and-Fire for End-to-End Speech Recognition](https://mlsanity.com/paper/1905.11235/) (2019)
- [WeNet 2.0: More Productive End-to-End Speech Recognition Toolkit](https://mlsanity.com/paper/2203.15455/) (2022)
- [Deep-FSMN for Large Vocabulary Continuous Speech Recognition](https://mlsanity.com/paper/1803.05030/) (2018)
- [Controllable Time-Delay Transformer for Real-Time Punctuation Prediction and Disfluency Detection](https://mlsanity.com/paper/2003.01309/) (2020)
- [SAN-M: Memory Equipped Self-Attention for End-to-End Speech Recognition](https://mlsanity.com/paper/2006.01713/) (2020)
- [Streaming Chunk-Aware Multihead Attention for Online End-to-End Speech Recognition](https://mlsanity.com/paper/2006.01712/) (2020)
- [Fast and parallel decoding for transducer](https://mlsanity.com/paper/2211.00484/) (2022)
- [DISC: A Dynamic Shape Compiler for Machine Learning Workloads](https://mlsanity.com/paper/2103.05288/) (2021)
- [Achieving Timestamp Prediction While Recognizing with Non-Autoregressive End-to-End ASR Model](https://mlsanity.com/paper/2301.12343/) (2023)
- [Universal ASR: Unifying Streaming and Non-Streaming ASR Using a Single Encoder-Decoder Model](https://mlsanity.com/paper/2010.14099/) (2020)
- [PaddleSpeech: An Easy-to-Use All-in-One Speech Toolkit](https://mlsanity.com/paper/2205.12007/) (2022)
- [ConvRNN-T: Convolutional Augmented Recurrent Neural Network Transducers for Streaming Speech Recognition](https://mlsanity.com/paper/2209.14868/) (2022)
- [Extremely Low Footprint End-to-End ASR System for Smart Device](https://mlsanity.com/paper/2104.05784/) (2021)
- [Memory-Efficient Training of RNN-Transducer with Sampled Softmax](https://mlsanity.com/paper/2203.16868/) (2022)
- [Separator-Transducer-Segmenter: Streaming Recognition and Segmentation of Multi-party Speech](https://mlsanity.com/paper/2205.05199/) (2022)
- [Multiple-hypothesis RNN-T Loss for Unsupervised Fine-tuning and Self-training of Neural Transducer](https://mlsanity.com/paper/2207.14736/) (2022)

## Built on by (most cited first, 30 of 95)

- [Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models](https://mlsanity.com/paper/2311.07919/) (2023)
- [Qwen2-Audio Technical Report](https://mlsanity.com/paper/2407.10759/) (2024)
- [F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching](https://mlsanity.com/paper/2410.06885/) (2024)
- [CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-training](https://mlsanity.com/paper/2505.17589/) (2025)
- [FunAudioLLM: Voice Understanding and Generation Foundation Models for Natural Interaction Between Humans and LLMs](https://mlsanity.com/paper/2407.04051/) (2024)
- [Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokens](https://mlsanity.com/paper/2503.01710/) (2025)
- [IndexTTS2: A Breakthrough in Emotionally Expressive and Duration-Controlled Auto-Regressive Zero-Shot Text-to-Speech](https://mlsanity.com/paper/2506.21619/) (2025)
- [LauraGPT: Listen, Attend, Understand, and Regenerate Audio with GPT](https://mlsanity.com/paper/2310.04673/) (2023)
- [Llasa: Scaling Train-Time and Inference-Time Compute for Llama-based Speech Synthesis](https://mlsanity.com/paper/2502.04128/) (2025)
- [OmniFlatten: An End-to-end GPT Model for Seamless Voice Conversation](https://mlsanity.com/paper/2410.17799/) (2024)
- [Uni-Sign: Toward Unified Sign Language Understanding at Scale](https://mlsanity.com/paper/2501.15187/) (2025)
- [InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions](https://mlsanity.com/paper/2412.09596/) (2024)
- [SongGen: A Single Stage Auto-regressive Transformer for Text-to-Song Generation](https://mlsanity.com/paper/2502.13128/) (2025)
- [SplitMeanFlow: Interval Splitting Consistency in Few-Step Generative Modeling](https://mlsanity.com/paper/2507.16884/) (2025)
- [JavisDiT++: Unified Modeling and Optimization for Joint Audio-Video Generation](https://mlsanity.com/paper/2602.19163/) (2026)
- [F5R-TTS: Improving Flow-Matching based Text-to-Speech with Group Relative Policy Optimization](https://mlsanity.com/paper/2504.02407/) (2025)
- [Speech Translation with Speech Foundation Models and Large Language Models: What is There and What is Missing?](https://mlsanity.com/paper/2402.12025/) (2024)
- [SeACo-Paraformer: A Non-Autoregressive ASR System with Flexible and Effective Hotword Customization Ability](https://mlsanity.com/paper/2308.03266/) (2023)
- [ContextASR-Bench: A Massive Contextual Speech Recognition Benchmark](https://mlsanity.com/paper/2507.05727/) (2025)
- [NVSpeech: An Integrated and Scalable Pipeline for Human-Like Speech Modeling with Paralinguistic Vocalizations](https://mlsanity.com/paper/2508.04195/) (2025)
- [SpeechJudge: Towards Human-Level Judgment for Speech Naturalness](https://mlsanity.com/paper/2511.07931/) (2025)
- [JoyAI-VL-Interaction: Real-Time Vision-Language Interaction Intelligence](https://mlsanity.com/paper/2606.14777/) (2026)
- [TaDiCodec: Text-aware Diffusion Speech Tokenizer for Speech Language Modeling](https://mlsanity.com/paper/2508.16790/) (2025)
- [SocialOmni: Benchmarking Audio-Visual Social Interactivity in Omni Models](https://mlsanity.com/paper/2603.16859/) (2026)
- [AudioGen-Omni: A Unified Multimodal Diffusion Transformer for Video-Synchronized Audio, Speech, and Song Generation](https://mlsanity.com/paper/2508.00733/) (2025)
- [FLM-Audio: Natural Monologues Improves Native Full-Duplex Chatbots via Dual Training](https://mlsanity.com/paper/2509.02521/) (2025)
- [LiveTalk: Real-Time Multimodal Interactive Video Diffusion via Improved On-Policy Distillation](https://mlsanity.com/paper/2512.23576/) (2025)
- [Fish Audio S2 Technical Report](https://mlsanity.com/paper/2603.08823/) (2026)
- [An Effective Mixture-Of-Experts Approach For Code-Switching Speech Recognition Leveraging Encoder Disentanglement](https://mlsanity.com/paper/2402.17189/) (2024)
- [DMOSpeech 2: Reinforcement Learning for Duration Prediction in Metric-Optimized Speech Synthesis](https://mlsanity.com/paper/2507.14988/) (2025)

## Abstract (arXiv)

This paper introduces FunASR, an open-source speech recognition toolkit designed to bridge the gap between academic research and industrial applications. FunASR offers models trained on large-scale industrial corpora and the ability to deploy them in applications. The toolkit's flagship model, Paraformer, is a non-autoregressive end-to-end speech recognition model that has been trained on a manually annotated Mandarin speech recognition dataset that contains 60,000 hours of speech. To improve the performance of Paraformer, we have added timestamp prediction and hotword customization capabilities to the standard Paraformer backbone. In addition, to facilitate model deployment, we have open-sourced a voice activity detection model based on the Feedforward Sequential Memory Network (FSMN-VAD) and a text post-processing punctuation model based on the controllable time-delay Transformer (CT-Transformer), both of which were trained on industrial corpora. These functional modules provide a solid foundation for building high-precision long audio speech recognition services. Compared to other models trained on open datasets, Paraformer demonstrates superior performance.