mlsanityData to 7 Oct 2026

Improving the Training Recipe for a Robust Conformer-based Hybrid Model

arXiv 2206.12955 ↗First posted 26 Jun 2022cs.CL eess.AS stat.MLcited by 9

Not on the map. Why?

Abstract (arXiv)

Speaker adaptation is important to build robust automatic speech recognition (ASR) systems. In this work, we investigate various methods for speaker adaptive training (SAT) based on feature-space approaches for a conformer-based acoustic model (AM) on the Switchboard 300h dataset. We propose a method, called Weighted-Simple-Add, which adds weighted speaker information vectors to the input of the multi-head self-attention module of the conformer AM. Using this method for SAT, we achieve 3.5% and 4.5% relative improvement in terms of WER on the CallHome part of Hub5'00 and Hub5'01 respectively. Moreover, we build on top of our previous work where we proposed a novel and competitive training recipe for a conformer-based hybrid AM. We extend and improve this recipe where we achieve 11% relative improvement in terms of word-error-rate (WER) on Switchboard 300h Hub5'00 dataset. We also make this recipe efficient by reducing the total number of parameters by 34% relative.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 23 of its 42 references that are in the map. First posted on arXiv 26 Jun 2022.

Builds on

23 of 42 references.

References · 23

  1. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  2. 2016Layer Normalizationcited by 5,579
  3. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  4. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  5. 2018Attentive Statistics Pooling for Deep Speaker Embeddingcited by 108
  6. 2019RWTH ASR Systems for LibriSpeech: Hybrid vs Attention -- w/o Data Augmentationcited by 63
  7. 2021Advancing RNN Transducer Technology for Speech Recognitioncited by 44
  8. 2020Single headed attention based sequence-to-sequence model for state-of-the-art results on Switchboardcited by 36
  9. 2018RETURNN as a Generic Flexible Neural Toolkit with Application to Translation and Speech Recognitioncited by 30
  10. 2021On the limit of English conversational speech recognitioncited by 30
  11. 2018Self-Attentional Acoustic Modelscited by 29
  12. 2020The RWTH ASR System for TED-LIUM Release 2: Improving Hybrid HMM with SpecAugmentcited by 19
11 more
  1. 2021Conformer-based Hybrid ASR System for Switchboard Datasetcited by 15
  2. 2021Improving Accent Identification and Accented Speech Recognition Under a Framework of Self-supervised Learningcited by 12
  3. 2022Layer-wise Fast Adaptation for End-to-End Multi-Accent Speech Recognitioncited by 12
  4. 2019Cumulative Adaptation for BLSTM Acoustic Modelscited by 11
  5. 2019LSTM Language Models for LVCSR in First-Pass Decoding and Lattice-Rescoringcited by 11
  6. 2020Multi-task Learning for Speaker Verification and Voice Trigger Detectioncited by 9
  7. 2020Multi-Accent Adaptation based on Gate Mechanismcited by 9
  8. 2020Bayesian Learning of LF-MMI Trained Time Delay Neural Networks for Speech Recognitioncited by 7
  9. 2022A Conformer Based Acoustic Model for Robust Automatic Speech Recognitioncited by 6
  10. 2019Embeddings for DNN speaker adaptive trainingcited by 4
  11. 2022A Highly Adaptive Acoustic Model for Accurate Multi-Dialect Speech Recognitioncited by 3

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 5; 2024: 1.

9 papers.

  1. 2022Enhancing and Adversarial: Improve ASR with Speaker Labelscited by 6
  2. 2023Towards Effective and Compact Contextual Representation for Conformer Transducer Speech Recognition Systemscited by 6
  3. 2022Uconv-Conformer: High Reduction of Input Sequence Length for End-to-End Speech Recognitioncited by 5
  4. 2022Development of Hybrid ASR Systems for Low Resource Medical Domain Conversational Telephone Speechcited by 3
  5. 2023Unsupervised Pre-Training for Vietnamese Automatic Speech Recognition in the HYKIST Projectcited by 2
  6. 2024Homogeneous Speaker Features for On-the-Fly Dysarthric and Elderly Speaker Adaptationcited by 2
  7. 2023End-to-End Training of a Neural HMM with Label and Transition Probabilitiescited by 1
  8. 2023Analyzing And Improving Neural Speaker Embeddings for ASRcited by 0
  9. 2023Factorised Speaker-environment Adaptive Training of Conformer Speech Recognition Systemscited by 0