# Sequence-to-sequence Automatic Speech Recognition with Word Embedding Regularization and Fused Decoding

arXiv: https://arxiv.org/abs/1910.12740 · HTML: https://mlsanity.com/paper/1910.12740/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Oct 2019.

## Built on by (most cited first, 3 of 3)

- [Improving CTC-based speech recognition via knowledge transferring from pre-trained language models](https://mlsanity.com/paper/2203.03582/) (2022)
- [Multiple Representation Transfer from Large Language Models to End-to-End ASR Systems](https://mlsanity.com/paper/2309.04031/) (2023)
- [Speech-Based Depression Prediction Using Encoder-Weight-Only Transfer Learning and a Large Corpus](https://mlsanity.com/paper/2412.16900/) (2024)

## Abstract (arXiv)

In this paper, we investigate the benefit that off-the-shelf word embedding can bring to the sequence-to-sequence (seq-to-seq) automatic speech recognition (ASR). We first introduced the word embedding regularization by maximizing the cosine similarity between a transformed decoder feature and the target word embedding. Based on the regularized decoder, we further proposed the fused decoding mechanism. This allows the decoder to consider the semantic consistency during decoding by absorbing the information carried by the transformed decoder feature, which is learned to be close to the target word embedding. Initial results on LibriSpeech demonstrated that pre-trained word embedding can significantly lower ASR recognition error with a negligible cost, and the choice of word embedding algorithms among Skip-gram, CBOW and BERT is important.