mlsanityData to 7 Oct 2026

Overcoming Domain Mismatch in Low Resource Sequence-to-Sequence ASR Models using Hybrid Generated Pseudotranscripts

arXiv 2106.07716 ↗First posted 14 Jun 2021cs.CL cs.SD eess.AScited by 3

Not on the map. Why?

Abstract (arXiv)

Sequence-to-sequence (seq2seq) models are competitive with hybrid models for automatic speech recognition (ASR) tasks when large amounts of training data are available. However, data sparsity and domain adaptation are more problematic for seq2seq models than their hybrid counterparts. We examine corpora of five languages from the IARPA MATERIAL program where the transcribed data is conversational telephone speech (CTS) and evaluation data is broadcast news (BN). We show that there is a sizable initial gap in such a data condition between hybrid and seq2seq models, and the hybrid model is able to further improve through the use of additional language model (LM) data. We use an additional set of untranscribed data primarily in the BN domain for semisupervised training. In semisupervised training, a seed model trained on transcribed data generates hypothesized transcripts for unlabeled domain-matched data for further training. By using a hybrid model with an expanded language model for pseudotranscription, we are able to improve our seq2seq model from an average word error rate (WER) of 66.7% across all five languages to 29.0% WER. While this puts the seq2seq model at a competitive operating point, hybrid models are still able to use additional LM data to maintain an advantage.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 18 of its 34 references that are in the map. First posted on arXiv 14 Jun 2021.

Builds on

18 of 34 references.

References · 18

  1. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  3. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  4. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  5. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 632
  6. 2015On Using Monolingual Corpora in Neural Machine Translationcited by 149
  7. 2020Improved Noisy Student Training for Automatic Speech Recognitioncited by 139
  8. 2019Self-Training for End-to-End Speech Recognitioncited by 128
  9. 2020Self-training and Pre-training are Complementary for Speech Recognitioncited by 98
  10. 2017Cold Fusion: Training Seq2Seq Models Together with Language Modelscited by 87
  11. 2018A Comparison of Techniques for Language Model Integration in Encoder-Decoder Speech Recognitioncited by 66
  12. 2020Single headed attention based sequence-to-sequence model for state-of-the-art results on Switchboardcited by 36
6 more
  1. 2018Improved training of end-to-end attention models for speech recognitioncited by 35
  2. 2018Multilingual sequence-to-sequence speech recognition: architecture, transfer learning, and language modelingcited by 34
  3. 2019Espresso: A Fast End-to-end Neural Speech Recognition Toolkitcited by 31
  4. 2020Semi-supervised ASR by End-to-end Self-trainingcited by 29
  5. 2020Improving Cross-Lingual Transfer Learning for End-to-End Speech Recognition with Speech Translationcited by 14
  6. 2018Transfer learning of language-independent end-to-end ASR with language model fusioncited by 10

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 2.

3 papers.

  1. 2022Unsupervised domain adaptation for speech recognition with unsupervised error correctioncited by 5
  2. 2021Combining Unsupervised and Text Augmented Semi-Supervised Learning for Low Resourced Autoregressive Speech Recognitioncited by 2
  3. 2022Training Autoregressive Speech Recognition Models with Limited in-domain Supervisioncited by 0