mlsanityData to 7 Oct 2026

Acoustic Model Fusion for End-to-end Speech Recognition

arXiv 2310.07062 ↗First posted 10 Oct 2023cs.SD cs.LG eess.AScited by 6

Not on the map. Why?

Abstract (arXiv)

Recent advances in deep learning and automatic speech recognition (ASR) have enabled the end-to-end (E2E) ASR system and boosted the accuracy to a new level. The E2E systems implicitly model all conventional ASR components, such as the acoustic model (AM) and the language model (LM), in a single network trained on audio-text pairs. Despite this simpler system architecture, fusing a separate LM, trained exclusively on text corpora, into the E2E system has proven to be beneficial. However, the application of LM fusion presents certain drawbacks, such as its inability to address the domain mismatch issue inherent to the internal AM. Drawing inspiration from the concept of LM fusion, we propose the integration of an external AM into the E2E system to better address the domain mismatch. By implementing this novel approach, we have achieved a significant reduction in the word error rate, with an impressive drop of up to 14.3% across varied test sets. We also discovered that this AM fusion approach is particularly beneficial in enhancing named entity recognition.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 17 of its 28 references that are in the map. First posted on arXiv 10 Oct 2023.

Builds on

17 of 28 references.

References · 17

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  4. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  5. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  6. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  7. 2017Self-Normalizing Neural Networkscited by 591
  8. 2015On Using Monolingual Corpora in Neural Machine Translationcited by 149
  9. 2020Recent Developments on ESPnet Toolkit Boosted by Conformercited by 132
  10. 2021WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkitcited by 123
  11. 2017An analysis of incorporating an external language model into a sequence-to-sequence modelcited by 109
  12. 2017Cold Fusion: Training Seq2Seq Models Together with Language Modelscited by 87
5 more
  1. 2020Internal Language Model Estimation for Domain-Adaptive End-to-End Speech Recognitioncited by 80
  2. 2020A Density Ratio Approach to Language Model Fusion in End-To-End Automatic Speech Recognitioncited by 73
  3. 2018A Comparison of Techniques for Language Model Integration in Encoder-Decoder Speech Recognitioncited by 66
  4. 2022WeNet 2.0: More Productive End-to-End Speech Recognition Toolkitcited by 46
  5. 2019SNDCNN: Self-normalizing deep CNNs with scaled exponential linear units for speech recognitioncited by 13

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 3; 2025: 2.

6 papers.

  1. 2024Contextualization of ASR with LLM using phonetic retrieval-based augmentationcited by 10
  2. 2023Personalization of CTC-based End-to-End Speech Recognition Using Pronunciation-Driven Subword Tokenizationcited by 6
  3. 2024Transformer-based Model for ASR N-Best Rescoring and Rewritingcited by 3
  4. 2025Re-evaluating Minimum Bayes Risk Decoding for Automatic Speech Recognitioncited by 2
  5. 2024Enhancing CTC-based speech recognition with diverse modeling unitscited by 1
  6. 2025Phonetically-Augmented Discriminative Rescoring for Voice Search Error Correctioncited by 0