# Contextual Biasing of Language Models for Speech Recognition in Goal-Oriented Conversational Agents

arXiv: https://arxiv.org/abs/2103.10325 · HTML: https://mlsanity.com/paper/2103.10325/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 17 references that are in the map.
- First posted on arXiv 18 Mar 2021.

## Builds on (most cited first, 11 of 11)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Dataset](https://mlsanity.com/paper/1909.05855/) (2019)
- [Sharp Nearby, Fuzzy Far Away: How Neural Language Models Use Context](https://mlsanity.com/paper/1805.04623/) (2018)
- [Dialog-context aware end-to-end speech recognition](https://mlsanity.com/paper/1808.02171/) (2018)
- [Personalized Language Model for Query Auto-Completion](https://mlsanity.com/paper/1804.09661/) (2018)
- [Contextual Language Model Adaptation for Conversational Agents](https://mlsanity.com/paper/1806.10215/) (2018)
- [Coherent Dialogue with Attention-based Language Models](https://mlsanity.com/paper/1611.06997/) (2016)
- [Long-span language modeling for speech recognition](https://mlsanity.com/paper/1911.04571/) (2019)
- [Dialog Context Language Modeling with Recurrent Neural Networks](https://mlsanity.com/paper/1701.04056/) (2017)
- [Context Dependent RNNLM for Automatic Transcription of Conversations](https://mlsanity.com/paper/2008.03517/) (2020)

## Built on by (most cited first, 4 of 4)

- [CUE Vectors: Modular Training of Language Models Conditioned on Diverse Contextual Signals](https://mlsanity.com/paper/2203.08774/) (2022)
- [Using External Off-Policy Speech-To-Text Mappings in Contextual End-To-End Automated Speech Recognition](https://mlsanity.com/paper/2301.02736/) (2023)
- [Improved Contextual Recognition In Automatic Speech Recognition Systems By Semantic Lattice Rescoring](https://mlsanity.com/paper/2310.09680/) (2023)
- [An Efficient Self-Learning Framework For Interactive Spoken Dialog Systems](https://mlsanity.com/paper/2409.10515/) (2024)

## Abstract (arXiv)

Goal-oriented conversational interfaces are designed to accomplish specific tasks and typically have interactions that tend to span multiple turns adhering to a pre-defined structure and a goal. However, conventional neural language models (NLM) in Automatic Speech Recognition (ASR) systems are mostly trained sentence-wise with limited context. In this paper, we explore different ways to incorporate context into a LSTM based NLM in order to model long range dependencies and improve speech recognition. Specifically, we use context carry over across multiple turns and use lexical contextual cues such as system dialog act from Natural Language Understanding (NLU) models and the user provided structure of the chatbot. We also propose a new architecture that utilizes context embeddings derived from BERT on sample utterances provided during inference time. Our experiments show a word error rate (WER) relative reduction of 7% over non-contextual utterance-level NLM rescorers on goal-oriented audio datasets.