# End-to-End Training of a Neural HMM with Label and Transition Probabilities

arXiv: https://arxiv.org/abs/2310.02724 · HTML: https://mlsanity.com/paper/2310.02724/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 37 references that are in the map.
- First posted on arXiv 4 Oct 2023.

## Builds on (most cited first, 11 of 11)

- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Cyclical Learning Rates for Training Neural Networks](https://mlsanity.com/paper/1506.01186/) (2015)
- [Transformer-based Acoustic Modeling for Hybrid Speech Recognition](https://mlsanity.com/paper/1910.09799/) (2019)
- [RETURNN: The RWTH Extensible Training framework for Universal Recurrent Neural Networks](https://mlsanity.com/paper/1608.00895/) (2016)
- [Efficient Training of Neural Transducer for Speech Recognition](https://mlsanity.com/paper/2204.10586/) (2022)
- [Improving the Training Recipe for a Robust Conformer-based Hybrid Model](https://mlsanity.com/paper/2206.12955/) (2022)
- [On lattice-free boosted MMI training of HMM and CTC-based full-context ASR models](https://mlsanity.com/paper/2107.04154/) (2021)
- [HMM vs. CTC for Automatic Speech Recognition: Comparison Based on Full-Sum Training from Scratch](https://mlsanity.com/paper/2210.09951/) (2022)
- [Context-Dependent Acoustic Modeling without Explicit Phone Clustering](https://mlsanity.com/paper/2005.07578/) (2020)
- [Improving Frame-level Classifier for Word Timings with Non-peaky CTC in End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/2306.07949/) (2023)
- [Competitive and Resource Efficient Factored Hybrid HMM Systems are Simpler Than You Think](https://mlsanity.com/paper/2306.09517/) (2023)

## Built on by (most cited first, 1 of 1)

- [Tempering the Bayes Filter towards Improved Model-Based Estimation](https://mlsanity.com/paper/2512.02823/) (2025)

## Abstract (arXiv)

We investigate a novel modeling approach for end-to-end neural network training using hidden Markov models (HMM) where the transition probabilities between hidden states are modeled and learned explicitly. Most contemporary sequence-to-sequence models allow for from-scratch training by summing over all possible label segmentations in a given topology. In our approach there are explicit, learnable probabilities for transitions between segments as opposed to a blank label that implicitly encodes duration statistics. We implement a GPU-based forward-backward algorithm that enables the simultaneous training of label and transition probabilities. We investigate recognition results and additionally Viterbi alignments of our models. We find that while the transition model training does not improve recognition performance, it has a positive impact on the alignment quality. The generated alignments are shown to be viable targets in state-of-the-art Viterbi trainings.