# Unidirectional Memory-Self-Attention Transducer for Online Speech Recognition

arXiv: https://arxiv.org/abs/2102.11594 · HTML: https://mlsanity.com/paper/2102.11594/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Feb 2021.

## Built on by (most cited first, 1 of 1)

- [Speech Augmentation Based Unsupervised Learning for Keyword Spotting](https://mlsanity.com/paper/2205.14329/) (2022)

## Abstract (arXiv)

Self-attention models have been successfully applied in end-to-end speech recognition systems, which greatly improve the performance of recognition accuracy. However, such attention-based models cannot be used in online speech recognition, because these models usually have to utilize a whole acoustic sequences as inputs. A common method is restricting the field of attention sights by a fixed left and right window, which makes the computation costs manageable yet also introduces performance degradation. In this paper, we propose Memory-Self-Attention (MSA), which adds history information into the Restricted-Self-Attention unit. MSA only needs localtime features as inputs, and efficiently models long temporal contexts by attending memory states. Meanwhile, recurrent neural network transducer (RNN-T) has proved to be a great approach for online ASR tasks, because the alignments of RNN-T are local and monotonic. We propose a novel network structure, called Memory-Self-Attention (MSA) Transducer. Both encoder and decoder of the MSA Transducer contain the proposed MSA unit. The experiments demonstrate that our proposed models improve WER results than Restricted-Self-Attention models by 13.5 on WSJ and 7.1 on SWBD datasets relatively, and without much computation costs increase.