mlsanityData to 7 Oct 2026

BayesSpeech: A Bayesian Transformer Network for Automatic Speech Recognition

arXiv 2301.11276 ↗First posted 16 Jan 2023eess.AS cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent developments using End-to-End Deep Learning models have been shown to have near or better performance than state of the art Recurrent Neural Networks (RNNs) on Automatic Speech Recognition tasks. These models tend to be lighter weight and require less training time than traditional RNN-based approaches. However, these models take frequentist approach to weight training. In theory, network weights are drawn from a latent, intractable probability distribution. We introduce BayesSpeech for end-to-end Automatic Speech Recognition. BayesSpeech is a Bayesian Transformer Network where these intractable posteriors are learned through variational inference and the local reparameterization trick without recurrence. We show how the introduction of variance in the weights leads to faster training time and near state-of-the-art performance on LibriSpeech-960.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 18 of its 31 references that are in the map. First posted on arXiv 16 Jan 2023.

Builds on

18 of 31 references.

References · 18

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  5. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  6. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  7. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  8. 2015Weight Uncertainty in Neural Networkscited by 554
  9. 2015Listen, Attend and Spellcited by 542
  10. 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 496
  11. 2015Variational Dropout and the Local Reparameterization Trickcited by 489
  12. 2015Attention-Based Models for Speech Recognitioncited by 400
6 more
  1. 2019An Unsupervised Autoregressive Model for Speech Representation Learningcited by 185
  2. 2019Jasper: An End-to-End Convolutional Neural Acoustic Modelcited by 75
  3. 2020Bayesian Attention Modulescited by 51
  4. 2022Towards End-to-end Unsupervised Speech Recognitioncited by 44
  5. 2017Advances in Joint CTC-Attention based End-to-End Speech Recognition with a Deep CNN Encoder and RNN-LMcited by 43
  6. 2021Bayesian Attention Belief Networkscited by 25

Built on by

Not cited yet.