# LT-LM: a novel non-autoregressive language model for single-shot lattice rescoring

arXiv: https://arxiv.org/abs/2104.02526 · HTML: https://mlsanity.com/paper/2104.02526/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 31 references that are in the map.
- First posted on arXiv 6 Apr 2021.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Training Deep Nets with Sublinear Memory Cost](https://mlsanity.com/paper/1604.06174/) (2016)
- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [Convolutional Sequence to Sequence Learning](https://mlsanity.com/paper/1705.03122/) (2017)
- [FLAT: Chinese NER Using Flat-Lattice Transformer](https://mlsanity.com/paper/2004.11795/) (2020)
- [A Comparison of Techniques for Language Model Integration in Encoder-Decoder Speech Recognition](https://mlsanity.com/paper/1807.10857/) (2018)
- [RWTH ASR Systems for LibriSpeech: Hybrid vs Attention -- w/o Data Augmentation](https://mlsanity.com/paper/1905.03072/) (2019)
- [English Conversational Telephone Speech Recognition by Humans and Machines](https://mlsanity.com/paper/1703.02136/) (2017)
- [Self-Attentional Models for Lattice Inputs](https://mlsanity.com/paper/1906.01617/) (2019)
- [A Parallelizable Lattice Rescoring Strategy with Neural Language Models](https://mlsanity.com/paper/2103.05081/) (2021)
- [The CAPIO 2017 Conversational Speech Recognition System](https://mlsanity.com/paper/1801.00059/) (2017)
- [Generating Human Readable Transcript for Automatic Speech Recognition with Pre-trained Language Model](https://mlsanity.com/paper/2102.11114/) (2021)

## Built on by (most cited first, 1 of 1)

- [Effective Cross-Utterance Language Modeling for Conversational Speech Recognition](https://mlsanity.com/paper/2111.03333/) (2021)

## Abstract (arXiv)

Neural network-based language models are commonly used in rescoring approaches to improve the quality of modern automatic speech recognition (ASR) systems. Most of the existing methods are computationally expensive since they use autoregressive language models. We propose a novel rescoring approach, which processes the entire lattice in a single call to the model. The key feature of our rescoring policy is a novel non-autoregressive Lattice Transformer Language Model (LT-LM). This model takes the whole lattice as an input and predicts a new language score for each arc. Additionally, we propose the artificial lattices generation approach to incorporate a large amount of text data in the LT-LM training process. Our single-shot rescoring performs orders of magnitude faster than other rescoring methods in our experiments. It is more than 300 times faster than pruned RNNLM lattice rescoring and N-best rescoring while slightly inferior in terms of WER.