# Listen Attentively, and Spell Once: Whole Sentence Generation via a Non-Autoregressive Architecture for Low-Latency Speech Recognition

arXiv: https://arxiv.org/abs/2005.04862 · HTML: https://mlsanity.com/paper/2005.04862/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 May 2020.

## Built on by (most cited first, 22 of 22)

- [Recent Advances in End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/2111.01690/) (2021)
- [A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond](https://mlsanity.com/paper/2204.09269/) (2022)
- [Relaxing the Conditional Independence Assumption of CTC-based ASR by Conditioning on Intermediate Predictions](https://mlsanity.com/paper/2104.02724/) (2021)
- [Fast End-to-End Speech Recognition via Non-Autoregressive Models and Cross-Modal Knowledge Transferring from BERT](https://mlsanity.com/paper/2102.07594/) (2021)
- [A Comparative Study on Non-Autoregressive Modelings for Speech-to-Text Generation](https://mlsanity.com/paper/2110.05249/) (2021)
- [PQK: Model Compression via Pruning, Quantization, and Knowledge Distillation](https://mlsanity.com/paper/2106.14681/) (2021)
- [Non-autoregressive Transformer-based End-to-end ASR using BERT](https://mlsanity.com/paper/2104.04805/) (2021)
- [An Improved Single Step Non-autoregressive Transformer for Automatic Speech Recognition](https://mlsanity.com/paper/2106.09885/) (2021)
- [TSNAT: Two-Step Non-Autoregressvie Transformer Models for Speech Recognition](https://mlsanity.com/paper/2104.01522/) (2021)
- [Decoupling recognition and transcription in Mandarin ASR](https://mlsanity.com/paper/2108.01129/) (2021)
- [Distilling the Knowledge of BERT for CTC-based ASR](https://mlsanity.com/paper/2209.02030/) (2022)
- [Non-autoregressive Error Correction for CTC-based ASR with Phone-conditioned Masked LM](https://mlsanity.com/paper/2209.04062/) (2022)
- [Non-autoregressive Transformer with Unified Bidirectional Decoder for Automatic Speech Recognition](https://mlsanity.com/paper/2109.06684/) (2021)
- [InterAug: Augmenting Noisy Intermediate Predictions for CTC-based ASR](https://mlsanity.com/paper/2204.00174/) (2022)
- [MLP-ASR: Sequence-length agnostic all-MLP architectures for speech recognition](https://mlsanity.com/paper/2202.08456/) (2022)
- [Dynamic Alignment Mask CTC: Improved Mask-CTC with Aligned Cross Entropy](https://mlsanity.com/paper/2303.07687/) (2023)
- [Sim-T: Simplify the Transformer Network by Multiplexing Technique for Speech Recognition](https://mlsanity.com/paper/2304.04991/) (2023)
- [Rethinking Speech Recognition with A Multimodal Perspective via Acoustic and Semantic Cooperative Decoding](https://mlsanity.com/paper/2305.14049/) (2023)
- [Exploring Non-Autoregressive End-To-End Neural Modeling For English Mispronunciation Detection And Diagnosis](https://mlsanity.com/paper/2111.00844/) (2021)
- [Towards Personalization of CTC Speech Recognition Models with Contextual Adapters and Adaptive Boosting](https://mlsanity.com/paper/2210.09510/) (2022)
- [Linguistic-Enhanced Transformer with CTC Embedding for Speech Recognition](https://mlsanity.com/paper/2210.14725/) (2022)
- [GNCformer Enhanced Self-attention for Automatic Speech Recognition](https://mlsanity.com/paper/2305.12755/) (2023)

## Abstract (arXiv)

Although attention based end-to-end models have achieved promising performance in speech recognition, the multi-pass forward computation in beam-search increases inference time cost, which limits their practical applications. To address this issue, we propose a non-autoregressive end-to-end speech recognition system called LASO (listen attentively, and spell once). Because of the non-autoregressive property, LASO predicts a textual token in the sequence without the dependence on other tokens. Without beam-search, the one-pass propagation much reduces inference time cost of LASO. And because the model is based on the attention based feedforward structure, the computation can be implemented in parallel efficiently. We conduct experiments on publicly available Chinese dataset AISHELL-1. LASO achieves a character error rate of 6.4%, which outperforms the state-of-the-art autoregressive transformer model (6.7%). The average inference latency is 21 ms, which is 1/50 of the autoregressive transformer model.