# Consistent Training and Decoding For End-to-end Speech Recognition Using Lattice-free MMI

arXiv: https://arxiv.org/abs/2112.02498 · HTML: https://mlsanity.com/paper/2112.02498/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 23 references that are in the map.
- First posted on arXiv 5 Dec 2021.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [Listen, Attend and Spell](https://mlsanity.com/paper/1508.01211/) (2015)
- [Minimum Word Error Rate Training for Attention-based Sequence-to-Sequence Models](https://mlsanity.com/paper/1712.01818/) (2017)
- [Joint Speaker Counting, Speech Recognition, and Speaker Identification for Overlapped Speech of Any Number of Speakers](https://mlsanity.com/paper/2006.10930/) (2020)
- [Efficient minimum word error rate training of RNN-Transducer for end-to-end speech recognition](https://mlsanity.com/paper/2007.13802/) (2020)
- [Non-Autoregressive Transformer ASR with CTC-Enhanced Decoder Input](https://mlsanity.com/paper/2010.15025/) (2020)
- [End-to-end Speech Recognition with Word-based RNN Language Models](https://mlsanity.com/paper/1808.02608/) (2018)
- [Minimum Bayes Risk Training of RNN-Transducer for End-to-End Speech Recognition](https://mlsanity.com/paper/1911.12487/) (2019)
- [Minimum Bayes Risk Training for End-to-End Speaker-Attributed ASR](https://mlsanity.com/paper/2011.02921/) (2020)
- [Multi-head Monotonic Chunkwise Attention For Online Speech Recognition](https://mlsanity.com/paper/2005.00205/) (2020)

## Built on by (most cited first, 8 of 8)

- [Improving Mandarin End-to-End Speech Recognition with Word N-gram Language Model](https://mlsanity.com/paper/2201.01995/) (2022)
- [Improving Mandarin Speech Recogntion with Block-augmented Transformer](https://mlsanity.com/paper/2207.11697/) (2022)
- [Improving CTC-based ASR Models with Gated Interlayer Collaboration](https://mlsanity.com/paper/2205.12462/) (2022)
- [Integrating Lattice-Free MMI into End-to-End Speech Recognition](https://mlsanity.com/paper/2203.15614/) (2022)
- [Lattice-Free Sequence Discriminative Training for Phoneme-Based Neural Transducers](https://mlsanity.com/paper/2212.04325/) (2022)
- [Multi-Level Modeling Units for End-to-End Mandarin Speech Recognition](https://mlsanity.com/paper/2205.11998/) (2022)
- [An Ensemble Teacher-Student Learning Approach with Poisson Sub-sampling to Differential Privacy Preserving Speech Recognition](https://mlsanity.com/paper/2210.06382/) (2022)
- [Investigating the Effect of Language Models in Sequence Discriminative Training for Neural Transducers](https://mlsanity.com/paper/2310.07345/) (2023)

## Abstract (arXiv)

Recently, End-to-End (E2E) frameworks have achieved remarkable results on various Automatic Speech Recognition (ASR) tasks. However, Lattice-Free Maximum Mutual Information (LF-MMI), as one of the discriminative training criteria that show superior performance in hybrid ASR systems, is rarely adopted in E2E ASR frameworks. In this work, we propose a novel approach to integrate LF-MMI criterion into E2E ASR frameworks in both training and decoding stages. The proposed approach shows its effectiveness on two of the most widely used E2E frameworks including Attention-Based Encoder-Decoders (AEDs) and Neural Transducers (NTs). Experiments suggest that the introduction of the LF-MMI criterion consistently leads to significant performance improvements on various datasets and different E2E ASR frameworks. The best of our models achieves competitive CER of 4.1% / 4.4% on Aishell-1 dev/test set; we also achieve significant error reduction on Aishell-2 and Librispeech datasets over strong baselines.