# Cross-Utterance Language Models with Acoustic Error Sampling

arXiv: https://arxiv.org/abs/2009.01008 · HTML: https://mlsanity.com/paper/2009.01008/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Aug 2020.

## Built on by (most cited first, 2 of 2)

- [Transformer Language Models with LSTM-based Cross-utterance Information Representation](https://mlsanity.com/paper/2102.06474/) (2021)
- [Leveraging Cross-Utterance Context For ASR Decoding](https://mlsanity.com/paper/2306.16903/) (2023)

## Abstract (arXiv)

The effective exploitation of richer contextual information in language models (LMs) is a long-standing research problem for automatic speech recognition (ASR). A cross-utterance LM (CULM) is proposed in this paper, which augments the input to a standard long short-term memory (LSTM) LM with a context vector derived from past and future utterances using an extraction network. The extraction network uses another LSTM to encode surrounding utterances into vectors which are integrated into a context vector using either a projection of LSTM final hidden states, or a multi-head self-attentive layer. In addition, an acoustic error sampling technique is proposed to reduce the mismatch between training and test-time. This is achieved by considering possible ASR errors into the model training procedure, and can therefore improve the word error rate (WER). Experiments performed on both AMI and Switchboard datasets show that CULMs outperform the LSTM LM baseline WER. In particular, the CULM with a self-attentive layer-based extraction network and acoustic error sampling achieves 0.6% absolute WER reduction on AMI, 0.3% WER reduction on the Switchboard part and 0.9% WER reduction on the Callhome part of Eval2000 test set over the respective baselines.