# On lattice-free boosted MMI training of HMM and CTC-based full-context ASR models

arXiv: https://arxiv.org/abs/2107.04154 · HTML: https://mlsanity.com/paper/2107.04154/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 43 references that are in the map.
- First posted on arXiv 9 Jul 2021.

## Builds on (most cited first, 20 of 20)

- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Speech Recognition with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1303.5778/) (2013)
- [Listen, Attend and Spell](https://mlsanity.com/paper/1508.01211/) (2015)
- [State-of-the-art Speech Recognition With Sequence-to-Sequence Models](https://mlsanity.com/paper/1712.01769/) (2017)
- [End-to-End Attention-based Large Vocabulary Speech Recognition](https://mlsanity.com/paper/1508.04395/) (2015)
- [Emformer: Efficient Memory Transformer Based Acoustic Model For Low Latency Streaming Speech Recognition](https://mlsanity.com/paper/2010.10759/) (2020)
- [Contextualized Streaming End-to-End Speech Recognition with Trie-Based Deep Biasing and Shallow Fusion](https://mlsanity.com/paper/2104.02194/) (2021)
- [Alignment Restricted Streaming Recurrent Neural Network Transducer](https://mlsanity.com/paper/2011.03072/) (2020)
- [A New Training Pipeline for an Improved Neural Transducer](https://mlsanity.com/paper/2005.09319/) (2020)
- [Why does CTC result in peaky behavior?](https://mlsanity.com/paper/2105.14849/) (2021)
- [Improving RNN Transducer Based ASR with Auxiliary Tasks](https://mlsanity.com/paper/2011.03109/) (2020)
- [From Senones to Chenones: Tied Context-Dependent Graphemes for Hybrid Speech Recognition](https://mlsanity.com/paper/1910.01493/) (2019)
- [PyChain: A Fully Parallelized PyTorch Implementation of LF-MMI for End-to-End ASR](https://mlsanity.com/paper/2005.09824/) (2020)
- [Benchmarking LF-MMI, CTC and RNN-T Criteria for Streaming ASR](https://mlsanity.com/paper/2011.04785/) (2020)
- [Faster, Simpler and More Accurate Hybrid ASR Systems Using Wordpieces](https://mlsanity.com/paper/2005.09150/) (2020)
- [Promising Accurate Prefix Boosting for sequence-to-sequence ASR](https://mlsanity.com/paper/1811.02770/) (2018)
- [Sequence Discriminative Training for Deep Learning based Acoustic Keyword Spotting](https://mlsanity.com/paper/1808.00639/) (2018)
- [A Comparison of Lattice-free Discriminative Training Criteria for Purely Sequence-Trained Neural Network Acoustic Models](https://mlsanity.com/paper/1811.03700/) (2018)

## Built on by (most cited first, 7 of 7)

- [Supervision-Guided Codebooks for Masked Prediction in Speech Pre-training](https://mlsanity.com/paper/2206.10125/) (2022)
- [Less Peaky and More Accurate CTC Forced Alignment by Label Priors](https://mlsanity.com/paper/2406.02560/) (2024)
- [Integrating Lattice-Free MMI into End-to-End Speech Recognition](https://mlsanity.com/paper/2203.15614/) (2022)
- [Lattice-Free Sequence Discriminative Training for Phoneme-Based Neural Transducers](https://mlsanity.com/paper/2212.04325/) (2022)
- [The Conformer Encoder May Reverse the Time Dimension](https://mlsanity.com/paper/2410.00680/) (2024)
- [End-to-End Training of a Neural HMM with Label and Transition Probabilities](https://mlsanity.com/paper/2310.02724/) (2023)
- [Investigating the Effect of Language Models in Sequence Discriminative Training for Neural Transducers](https://mlsanity.com/paper/2310.07345/) (2023)

## Abstract (arXiv)

Hybrid automatic speech recognition (ASR) models are typically sequentially trained with CTC or LF-MMI criteria. However, they have vastly different legacies and are usually implemented in different frameworks. In this paper, by decoupling the concepts of modeling units and label topologies and building proper numerator/denominator graphs accordingly, we establish a generalized framework for hybrid acoustic modeling (AM). In this framework, we show that LF-MMI is a powerful training criterion applicable to both limited-context and full-context models, for wordpiece/mono-char/bi-char/chenone units, with both HMM/CTC topologies. From this framework, we propose three novel training schemes: chenone(ch)/wordpiece(wp)-CTC-bMMI, and wordpiece(wp)-HMM-bMMI with different advantages in training performance, decoding efficiency and decoding time-stamp accuracy. The advantages of different training schemes are evaluated comprehensively on Librispeech, and wp-CTC-bMMI and ch-CTC-bMMI are evaluated on two real world ASR tasks to show their effectiveness. Besides, we also show bi-char(bc) HMM-MMI models can serve as better alignment models than traditional non-neural GMM-HMMs.