# Streaming Attention-Based Models with Augmented Memory for End-to-End Speech Recognition

arXiv: https://arxiv.org/abs/2011.07120 · HTML: https://mlsanity.com/paper/2011.07120/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Nov 2020.

## Built on by (most cited first, 4 of 4)

- [Dynamic Encoder Transducer: A Flexible Solution For Trading Off Accuracy For Latency](https://mlsanity.com/paper/2104.02176/) (2021)
- [Streaming Transformer Transducer Based Speech Recognition Using Non-Causal Convolution](https://mlsanity.com/paper/2110.05241/) (2021)
- [Flexi-Transducer: Optimizing Latency, Accuracy and Compute forMulti-Domain On-Device Scenarios](https://mlsanity.com/paper/2104.02232/) (2021)
- [Dynamic Latency for CTC-Based Streaming Automatic Speech Recognition With Emformer](https://mlsanity.com/paper/2203.15613/) (2022)

## Abstract (arXiv)

Attention-based models have been gaining popularity recently for their strong performance demonstrated in fields such as machine translation and automatic speech recognition. One major challenge of attention-based models is the need of access to the full sequence and the quadratically growing computational cost concerning the sequence length. These characteristics pose challenges, especially for low-latency scenarios, where the system is often required to be streaming. In this paper, we build a compact and streaming speech recognition system on top of the end-to-end neural transducer architecture with attention-based modules augmented with convolution. The proposed system equips the end-to-end models with the streaming capability and reduces the large footprint from the streaming attention-based model using augmented memory. On the LibriSpeech dataset, our proposed system achieves word error rates 2.7% on test-clean and 5.8% on test-other, to our best knowledge the lowest among streaming approaches reported so far.