mlsanityData to 14 Oct 2026

Input Length Matters: Improving RNN-T and MWER Training for Long-form Telephony Speech Recognition

arXiv 2110.03841 ↗First posted 8 Oct 2021eess.AS cs.CLcited by 10

Not on the map. Why?

Abstract (arXiv)

End-to-end models have achieved state-of-the-art results on several automatic speech recognition tasks. However, they perform poorly when evaluated on long-form data, e.g., minutes long conversational telephony audio. One reason the model fails on long-form speech is that it has only seen short utterances during training. In this paper we study the effect of training utterance length on the word error rate (WER) for RNN-transducer (RNN-T) model. We compare two widely used training objectives, log loss (or RNN-T loss) and minimum word error rate (MWER) loss. We conduct experiments on telephony datasets in four languages. Our experiments show that for both losses, the WER on long-form speech reduces substantially as the training utterance length increases. The average relative WER gain is 15.7% for log loss and 8.8% for MWER loss. When training on short utterances, MWER loss leads to a lower WER than the log loss. Such difference between the two losses diminishes when the input length increases.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 28 of its 33 references that are in the map. First posted on arXiv 8 Oct 2021.

Builds on

28 of 33 references.

References · 28

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  3. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,380
  4. 2013Speech Recognition with Deep Recurrent Neural Networkscited by 669
  5. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 618
  6. 2020Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognitioncited by 187
  7. 2021BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognitioncited by 88
  8. 2019Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modelingcited by 86
  9. 2019Recognizing long-form speech using streaming end-to-end modelscited by 77
  10. 2020A Better and Faster End-to-End Model for Streaming ASRcited by 69
  11. 2017Minimum Word Error Rate Training for Attention-based Sequence-to-Sequence Modelscited by 68
  12. 2019Improving RNN Transducer Modeling for End-to-End Speech Recognitioncited by 68
16 more
  1. 2017The Microsoft 2017 Conversational Speech Recognition Systemcited by 52
  2. 2021Advancing RNN Transducer Technology for Speech Recognitioncited by 44
  3. 2019A comparison of end-to-end models for long-form speech recognitioncited by 42
  4. 2020A New Training Pipeline for an Improved Neural Transducercited by 34
  5. 2020Efficient minimum word error rate training of RNN-Transducer for end-to-end speech recognitioncited by 34
  6. 2021On the limit of English conversational speech recognitioncited by 30
  7. 2020RNN-T Models Fail to Generalize to Out-of-Domain Audio: Causes and Solutionscited by 26
  8. 2020Less Is More: Improved RNN-T Decoding Using Limited Label Context and Path Mergingcited by 26
  9. 2021Advanced Long-context End-to-end Speech Recognition Using Context-expanded Transformerscited by 25
  10. 2019Minimum Bayes Risk Training of RNN-Transducer for End-to-End Speech Recognitioncited by 23
  11. 2017Optimizing expected word error rate via sampling for speech recognitioncited by 21
  12. 2020Improving Streaming Automatic Speech Recognition With Non-Streaming Model Distillation On Unsupervised Datacited by 16
  13. 2020On Minimum Word Error Rate Training of the Hybrid Autoregressive Transducercited by 11
  14. 2021Residual Energy-Based Models for End-to-End Speech Recognitioncited by 9
  15. 2021Bridging the gap between streaming and non-streaming ASR systems bydistilling ensembles of CTC and RNN-T modelscited by 3
  16. 2021Reducing Exposure Bias in Training Recurrent Neural Network Transducerscited by 2

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 8; 2024: 1.

10 papers.

  1. 2023Google USM: Scaling Automatic Speech Recognition Beyond 100 Languagescited by 231
  2. 2024How "Real" is Your Real-Time Simultaneous Speech-to-Text Translation System?cited by 17
  3. 2023Updated Corpora and Benchmarks for Long-Form Speech Recognitioncited by 10
  4. 2023USM-SCD: Multilingual Speaker Change Detection Based on Large Pretrained Foundation Modelscited by 4
  5. 2022Random Utterance Concatenation Based Data Augmentation for Improving Short-video Speech Recognitioncited by 3
  6. 2023Long-form Simultaneous Speech Translation: Thesis Proposalcited by 3
  7. 2023How Much Context Does My Attention-Based ASR System Need?cited by 3
  8. 2023DSS: Synthesizing long Digital Ink using Data augmentation, Style encoding and Split generationcited by 2
  9. 2023Improved Long-Form Speech Recognition by Jointly Modeling the Primary and Non-primary Speakerscited by 2
  10. 2023O-1: Self-training with Oracle and 1-best Hypothesiscited by 0