# A Study of Transducer based End-to-End ASR with ESPnet: Architecture, Auxiliary Loss and Decoding Strategies

arXiv: https://arxiv.org/abs/2201.05420 · HTML: https://mlsanity.com/paper/2201.05420/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Jan 2022.

## Built on by (most cited first, 15 of 15)

- [Foundation Transformers](https://mlsanity.com/paper/2210.06423/) (2022)
- [BERT Meets CTC: New Formulation of End-to-End Speech Recognition with Pre-trained Masked Language Model](https://mlsanity.com/paper/2210.16663/) (2022)
- [LAHAJA: A Robust Multi-accent Benchmark for Evaluating Hindi ASR Systems](https://mlsanity.com/paper/2408.11440/) (2024)
- [When Is TTS Augmentation Through a Pivot Language Useful?](https://mlsanity.com/paper/2207.09889/) (2022)
- [BECTRA: Transducer-based End-to-End ASR with BERT-Enhanced Encoder](https://mlsanity.com/paper/2211.00792/) (2022)
- [Minimum Latency Training of Sequence Transducers for Streaming End-to-End Speech Recognition](https://mlsanity.com/paper/2211.02333/) (2022)
- [Empowering Low-Resource Language ASR via Large-Scale Pseudo Labeling](https://mlsanity.com/paper/2408.14026/) (2024)
- [Memory-Efficient Training of RNN-Transducer with Sampled Softmax](https://mlsanity.com/paper/2203.16868/) (2022)
- [A Token-Wise Beam Search Algorithm for RNN-T](https://mlsanity.com/paper/2302.14357/) (2023)
- [Decoupled Structure for Improved Adaptability of End-to-End Models](https://mlsanity.com/paper/2308.13345/) (2023)
- [Token-Weighted RNN-T for Learning from Flawed Data](https://mlsanity.com/paper/2406.18108/) (2024)
- [Weak Alignment Supervision from Hybrid Model Improves End-to-end ASR](https://mlsanity.com/paper/2311.14835/) (2023)
- [Boosting Hybrid Autoregressive Transducer-based ASR with Internal Acoustic Model Training and Dual Blank Thresholding](https://mlsanity.com/paper/2409.20313/) (2024)
- [Globally Normalising the Transducer for Streaming Speech Recognition](https://mlsanity.com/paper/2307.10975/) (2023)
- [Alternating Weak Triphone/BPE Alignment Supervision from Hybrid Model Improves End-to-End ASR](https://mlsanity.com/paper/2402.15594/) (2024)

## Abstract (arXiv)

In this study, we present recent developments of models trained with the RNN-T loss in ESPnet. It involves the use of various architectures such as recently proposed Conformer, multi-task learning with different auxiliary criteria and multiple decoding strategies, including our own proposition. Through experiments and benchmarks, we show that our proposed systems can be competitive against other state-of-art systems on well-known datasets such as LibriSpeech and AISHELL-1. Additionally, we demonstrate that these models are promising against other already implemented systems in ESPnet in regards to both performance and decoding speed, enabling the possibility to have powerful systems for a streaming task. With these additions, we hope to expand the usefulness of the ESPnet toolkit for the research community and also give tools for the ASR industry to deploy our systems in realistic and production environments.