# WeNet 2.0: More Productive End-to-End Speech Recognition Toolkit

arXiv: https://arxiv.org/abs/2203.15455 · HTML: https://mlsanity.com/paper/2203.15455/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 36 references that are in the map.
- First posted on arXiv 29 Mar 2022.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Speech Recognition with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1303.5778/) (2013)
- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [Listen, Attend and Spell](https://mlsanity.com/paper/1508.01211/) (2015)
- [Deep Speech 2: End-to-End Speech Recognition in English and Mandarin](https://mlsanity.com/paper/1512.02595/) (2015)
- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [Attention-Based Models for Speech Recognition](https://mlsanity.com/paper/1506.07503/) (2015)
- [GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audio](https://mlsanity.com/paper/2106.06909/) (2021)
- [AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baseline](https://mlsanity.com/paper/1709.05522/) (2017)
- [Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learning](https://mlsanity.com/paper/1609.06773/) (2016)
- [Streaming End-to-end Speech Recognition For Mobile Devices](https://mlsanity.com/paper/1811.06621/) (2018)
- [WenetSpeech: A 10000+ Hours Multi-domain Mandarin Corpus for Speech Recognition](https://mlsanity.com/paper/2110.03370/) (2021)
- [Recent Advances in End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/2111.01690/) (2021)
- [AISHELL-2: Transforming Mandarin ASR Research Into Industrial Scale](https://mlsanity.com/paper/1808.10583/) (2018)
- [WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkit](https://mlsanity.com/paper/2102.01547/) (2021)
- [A Streaming On-Device End-to-End Model Surpassing Server-Side Conventional Model Quality and Latency](https://mlsanity.com/paper/2003.12710/) (2020)
- [Contextual RNN-T For Open Domain ASR](https://mlsanity.com/paper/2006.03411/) (2020)
- [End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results](https://mlsanity.com/paper/1412.1602/) (2014)
- [Deliberation Model Based Two-Pass End-to-End Speech Recognition](https://mlsanity.com/paper/2003.07962/) (2020)
- [Transformer-based Online CTC/attention End-to-End Speech Recognition Architecture](https://mlsanity.com/paper/2001.08290/) (2020)
- [High Performance I/O For Large Scale Deep Learning](https://mlsanity.com/paper/2001.01858/) (2020)
- [Simplified Self-Attention for Transformer-based End-to-End Speech Recognition](https://mlsanity.com/paper/2005.10463/) (2020)
- [Cascade RNN-Transducer: Syllable Based Streaming On-device Mandarin Speech Recognition with a Syllable-to-Character Converter](https://mlsanity.com/paper/2011.08469/) (2020)
- [Exploring RNN-Transducer for Chinese Speech Recognition](https://mlsanity.com/paper/1811.05097/) (2018)

## Built on by (most cited first, 30 of 47)

- [FunASR: A Fundamental End-to-End Speech Recognition Toolkit](https://mlsanity.com/paper/2305.11013/) (2023)
- [Zipformer: A faster and better encoder for automatic speech recognition](https://mlsanity.com/paper/2310.11230/) (2023)
- [Contextualized End-to-End Speech Recognition with Contextual Phrase Prediction Network](https://mlsanity.com/paper/2305.12493/) (2023)
- [CR-CTC: Consistency regularization on CTC for improved speech recognition](https://mlsanity.com/paper/2410.05101/) (2024)
- [ZeroPrompt: Streaming Acoustic Encoders are Zero-Shot Masked LMs](https://mlsanity.com/paper/2305.10649/) (2023)
- [Unveiling the Potential of LLM-Based ASR on Chinese Open-Source Datasets](https://mlsanity.com/paper/2405.02132/) (2024)
- [Adaptive Contextual Biasing for Transducer Based Streaming Speech Recognition](https://mlsanity.com/paper/2306.00804/) (2023)
- [SummaryMixing: A Linear-Complexity Alternative to Self-Attention for Speech Recognition and Understanding](https://mlsanity.com/paper/2307.07421/) (2023)
- [TouchTTS: An Embarrassingly Simple TTS Framework that Everyone Can Touch](https://mlsanity.com/paper/2412.08237/) (2024)
- [TrimTail: Low-Latency Streaming ASR with Simple but Effective Spectrogram-Level Length Penalty](https://mlsanity.com/paper/2211.00522/) (2022)
- [Acoustic Model Fusion for End-to-end Speech Recognition](https://mlsanity.com/paper/2310.07062/) (2023)
- [Conformer-Based Speech Recognition On Extreme Edge-Computing Devices](https://mlsanity.com/paper/2312.10359/) (2023)
- [Uni-ASR: Unified LLM-Based Architecture for Non-Streaming and Streaming Automatic Speech Recognition](https://mlsanity.com/paper/2603.11123/) (2026)
- [U2++ MoE: Scaling 4.7x parameters with minimal impact on RTF](https://mlsanity.com/paper/2404.16407/) (2024)
- [CNVSRC 2023: The First Chinese Continuous Visual Speech Recognition Challenge](https://mlsanity.com/paper/2406.10313/) (2024)
- [Leveraging LLM and Self-Supervised Training Models for Speech Recognition in Chinese Dialects: A Comparative Analysis](https://mlsanity.com/paper/2505.21138/) (2025)
- [Audio-to-Intent Using Acoustic-Textual Subword Representations from End-to-End ASR](https://mlsanity.com/paper/2210.12134/) (2022)
- [SF-Speech: Straightened Flow for Zero-Shot Voice Clone](https://mlsanity.com/paper/2410.12399/) (2024)
- [The Eloquence team submission for task 1 of MLC-SLM challenge](https://mlsanity.com/paper/2507.19308/) (2025)
- [RdimKD: Generic Distillation Paradigm by Dimensionality Reduction](https://mlsanity.com/paper/2312.08700/) (2023)
- [Temporal Order Preserved Optimal Transport-based Cross-modal Knowledge Transfer Learning for ASR](https://mlsanity.com/paper/2409.02239/) (2024)
- [REF-VC: Robust, Expressive and Fast Zero-Shot Voice Conversion with Diffusion Transformers](https://mlsanity.com/paper/2508.04996/) (2025)
- [Spatio-Temporal driven Attention Graph Neural Network with Block Adjacency matrix (STAG-NN-BA) for Remote Land-use Change Detection](https://mlsanity.com/paper/2303.14322/) (2023)
- [Key Frame Mechanism For Efficient Conformer Based End-to-end Speech Recognition](https://mlsanity.com/paper/2310.14954/) (2023)
- [UCorrect: An Unsupervised Framework for Automatic Speech Recognition Error Correction](https://mlsanity.com/paper/2401.05689/) (2024)
- [Qifusion-Net: Layer-adapted Stream/Non-stream Model for End-to-End Multi-Accent Speech Recognition](https://mlsanity.com/paper/2407.03026/) (2024)
- [Dynamic Language Group-Based MoE: Enhancing Code-Switching Speech Recognition with Hierarchical Routing](https://mlsanity.com/paper/2407.18581/) (2024)
- [Enhancing Code-Switching Speech Recognition with LID-Based Collaborative Mixture of Experts Model](https://mlsanity.com/paper/2409.02050/) (2024)
- [SSCFormer: Push the Limit of Chunk-wise Conformer for Streaming ASR Using Sequentially Sampled Chunks and Chunked Causal Convolution](https://mlsanity.com/paper/2211.11419/) (2022)
- [R-BI: Regularized Batched Inputs enhance Incremental Decoding Framework for Low-Latency Simultaneous Speech Translation](https://mlsanity.com/paper/2401.05700/) (2024)

## Abstract (arXiv)

Recently, we made available WeNet, a production-oriented end-to-end speech recognition toolkit, which introduces a unified two-pass (U2) framework and a built-in runtime to address the streaming and non-streaming decoding modes in a single model. To further improve ASR performance and facilitate various production requirements, in this paper, we present WeNet 2.0 with four important updates. (1) We propose U2++, a unified two-pass framework with bidirectional attention decoders, which includes the future contextual information by a right-to-left attention decoder to improve the representative ability of the shared encoder and the performance during the rescoring stage. (2) We introduce an n-gram based language model and a WFST-based decoder into WeNet 2.0, promoting the use of rich text data in production scenarios. (3) We design a unified contextual biasing framework, which leverages user-specific context (e.g., contact lists) to provide rapid adaptation ability for production and improves ASR accuracy in both with-LM and without-LM scenarios. (4) We design a unified IO to support large-scale data for effective model training. In summary, the brand-new WeNet 2.0 achieves up to 10% relative recognition performance improvement over the original WeNet on various corpora and makes available several important production-oriented features.