# StableEmit: Selection Probability Discount for Reducing Emission Latency of Streaming Monotonic Attention ASR

arXiv: https://arxiv.org/abs/2107.00635 · HTML: https://mlsanity.com/paper/2107.00635/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 31 references that are in the map.
- First posted on arXiv 1 Jul 2021.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [Correcting Length Bias in Neural Machine Translation](https://mlsanity.com/paper/1808.10006/) (2018)
- [Towards better decoding and language model integration in sequence to sequence models](https://mlsanity.com/paper/1612.02695/) (2016)
- [Online and Linear-Time Attention by Enforcing Monotonic Alignments](https://mlsanity.com/paper/1704.00784/) (2017)
- [Monotonic Chunkwise Attention](https://mlsanity.com/paper/1712.05382/) (2017)
- [FastEmit: Low-latency Streaming ASR with Sequence-level Emission Regularization](https://mlsanity.com/paper/2010.11148/) (2020)
- [Alignment Restricted Streaming Recurrent Neural Network Transducer](https://mlsanity.com/paper/2011.03072/) (2020)
- [Self-Attention Aligner: A Latency-Control End-to-End Model for ASR Using Self-Attention Network and Chunk-Hopping](https://mlsanity.com/paper/1902.06450/) (2019)
- [Minimum Latency Training Strategies for Streaming Sequence-to-Sequence ASR](https://mlsanity.com/paper/2004.05009/) (2020)
- [Reducing Streaming ASR Model Delay with Self Alignment](https://mlsanity.com/paper/2105.05005/) (2021)
- [A review of on-device fully neural end-to-end automatic speech recognition algorithms](https://mlsanity.com/paper/2012.07974/) (2020)
- [Alignment Knowledge Distillation for Online Streaming Attention-based Speech Recognition](https://mlsanity.com/paper/2103.00422/) (2021)
- [Exploring Pre-training with Alignments for RNN Transducer based End-to-End Speech Recognition](https://mlsanity.com/paper/2005.00572/) (2020)
- [CTC-synchronous Training for Monotonic Attention Model](https://mlsanity.com/paper/2005.04712/) (2020)

## Built on by (most cited first, 1 of 1)

- [VAD-free Streaming Hybrid CTC/Attention ASR for Unsegmented Recording](https://mlsanity.com/paper/2107.07509/) (2021)

## Abstract (arXiv)

While attention-based encoder-decoder (AED) models have been successfully extended to the online variants for streaming automatic speech recognition (ASR), such as monotonic chunkwise attention (MoChA), the models still have a large label emission latency because of the unconstrained end-to-end training objective. Previous works tackled this problem by leveraging alignment information to control the timing to emit tokens during training. In this work, we propose a simple alignment-free regularization method, StableEmit, to encourage MoChA to emit tokens earlier. StableEmit discounts the selection probabilities in hard monotonic attention for token boundary detection by a constant factor and regularizes them to recover the total attention mass during training. As a result, the scale of the selection probabilities is increased, and the values can reach a threshold for token emission earlier, leading to a reduction of emission latency and deletion errors. Moreover, StableEmit can be combined with methods that constraint alignments to further improve the accuracy and latency. Experimental evaluations with LSTM and Conformer encoders demonstrate that StableEmit significantly reduces the recognition errors and the emission latency simultaneously. We also show that the use of alignment information is complementary in both metrics.