# Espresso: A Fast End-to-end Neural Speech Recognition Toolkit

arXiv: https://arxiv.org/abs/1909.08723 · HTML: https://mlsanity.com/paper/1909.08723/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Sep 2019.

## Built on by (most cited first, 30 of 31)

- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [Recent Advances in End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/2111.01690/) (2021)
- [End-to-End Speech Recognition: A Survey](https://mlsanity.com/paper/2303.03329/) (2023)
- [Efficient Sequence Transduction by Jointly Predicting Tokens and Durations](https://mlsanity.com/paper/2304.06795/) (2023)
- [Wav2vec-Switch: Contrastive Learning from Original-noisy Speech Pairs for Robust Speech Recognition](https://mlsanity.com/paper/2110.04934/) (2021)
- [Semantic-WER: A Unified Metric for the Evaluation of ASR Transcript for End Usability](https://mlsanity.com/paper/2106.02016/) (2021)
- [Improving Noise Robustness of Contrastive Speech Representation Learning with Speech Reconstruction](https://mlsanity.com/paper/2110.15430/) (2021)
- [Sequential Randomized Smoothing for Adversarially Robust Speech Recognition](https://mlsanity.com/paper/2112.03000/) (2021)
- [Multi-blank Transducers for Speech Recognition](https://mlsanity.com/paper/2211.03541/) (2022)
- [End2End Acoustic to Semantic Transduction](https://mlsanity.com/paper/2102.01013/) (2021)
- [Wake Word Detection with Streaming Transformers](https://mlsanity.com/paper/2102.04488/) (2021)
- [Label-Looping: Highly Efficient Decoding for Transducers](https://mlsanity.com/paper/2406.06220/) (2024)
- [Multi-Channel Multi-Speaker ASR Using 3D Spatial Feature](https://mlsanity.com/paper/2111.11023/) (2021)
- [Multi-Encoder Learning and Stream Fusion for Transformer-Based End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/2104.00120/) (2021)
- [Comparing CTC and LFMMI for out-of-domain adaptation of wav2vec 2.0 acoustic model](https://mlsanity.com/paper/2104.02558/) (2021)
- [Improving Low-Resource Speech Recognition with Pretrained Speech Models: Continued Pretraining vs. Semi-Supervised Training](https://mlsanity.com/paper/2207.00659/) (2022)
- [Insights on Neural Representations for End-to-End Speech Recognition](https://mlsanity.com/paper/2205.09456/) (2022)
- [Relaxed Attention for Transformer Models](https://mlsanity.com/paper/2209.09735/) (2022)
- [Training Integer-Only Deep Recurrent Neural Networks](https://mlsanity.com/paper/2212.11791/) (2022)
- [Relaxed Attention: A Simple Method to Boost Performance of End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/2107.01275/) (2021)
- [Overcoming Domain Mismatch in Low Resource Sequence-to-Sequence ASR Models using Hybrid Generated Pseudotranscripts](https://mlsanity.com/paper/2106.07716/) (2021)
- [iRNN: Integer-only Recurrent Neural Network](https://mlsanity.com/paper/2109.09828/) (2021)
- [Quran Recitation Recognition using End-to-End Deep Learning](https://mlsanity.com/paper/2305.07034/) (2023)
- [Deformable TDNN with adaptive receptive fields for speech recognition](https://mlsanity.com/paper/2104.14791/) (2021)
- [BEA-Base: A Benchmark for ASR of Spontaneous Hungarian](https://mlsanity.com/paper/2202.00601/) (2022)
- [Effectiveness of Text, Acoustic, and Lattice-based representations in Spoken Language Understanding tasks](https://mlsanity.com/paper/2212.08489/) (2022)
- [Probing Statistical Representations For End-To-End ASR](https://mlsanity.com/paper/2211.01993/) (2022)
- [EFI: A Toolbox for Feature Importance Fusion and Interpretation in Python](https://mlsanity.com/paper/2208.04343/) (2022)
- [RIR-SF: Room Impulse Response Based Spatial Feature for Target Speech Recognition in Multi-Channel Multi-Speaker Scenarios](https://mlsanity.com/paper/2311.00146/) (2023)
- [Transducers with Pronunciation-aware Embeddings for Automatic Speech Recognition](https://mlsanity.com/paper/2404.04295/) (2024)

## Abstract (arXiv)

We present Espresso, an open-source, modular, extensible end-to-end neural automatic speech recognition (ASR) toolkit based on the deep learning library PyTorch and the popular neural machine translation toolkit fairseq. Espresso supports distributed training across GPUs and computing nodes, and features various decoding approaches commonly employed in ASR, including look-ahead word-based language model fusion, for which a fast, parallelized decoder is implemented. Espresso achieves state-of-the-art ASR performance on the WSJ, LibriSpeech, and Switchboard data sets among other end-to-end systems without data augmentation, and is 4--11x faster for decoding than similar systems (e.g. ESPnet).