# Scalable language model adaptation for spoken dialogue systems

arXiv: https://arxiv.org/abs/1812.04647 · HTML: https://mlsanity.com/paper/1812.04647/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Dec 2018.

## Built on by (most cited first, 11 of 11)

- [Adapting Long Context NLM for ASR Rescoring in Conversational Agents](https://mlsanity.com/paper/2104.11070/) (2021)
- [Synthetic Query Generation using Large Language Models for Virtual Assistants](https://mlsanity.com/paper/2406.06729/) (2024)
- [Error-driven Pruning of Language Models for Virtual Assistants](https://mlsanity.com/paper/2102.07219/) (2021)
- [ASR Adaptation for E-commerce Chatbots using Cross-Utterance Context and Multi-Task Language Modeling](https://mlsanity.com/paper/2106.09532/) (2021)
- [A Discriminative Entity-Aware Language Model for Virtual Assistants](https://mlsanity.com/paper/2106.11292/) (2021)
- [Space-Efficient Representation of Entity-centric Query Language Models](https://mlsanity.com/paper/2206.14885/) (2022)
- [A Likelihood Ratio based Domain Adaptation Method for E2E Models](https://mlsanity.com/paper/2201.03655/) (2022)
- [Using External Off-Policy Speech-To-Text Mappings in Contextual End-To-End Automated Speech Recognition](https://mlsanity.com/paper/2301.02736/) (2023)
- [Modeling Spoken Information Queries for Virtual Assistants: Open Problems, Challenges and Opportunities](https://mlsanity.com/paper/2304.13149/) (2023)
- [Dynamic Acoustic Unit Augmentation With BPE-Dropout for Low-Resource End-to-End Speech Recognition](https://mlsanity.com/paper/2103.07186/) (2021)
- [Adaptive Multi-Corpora Language Model Training for Speech Recognition](https://mlsanity.com/paper/2211.05121/) (2022)

## Abstract (arXiv)

Language models (LM) for interactive speech recognition systems are trained on large amounts of data and the model parameters are optimized on past user data. New application intents and interaction types are released for these systems over time, imposing challenges to adapt the LMs since the existing training data is no longer sufficient to model the future user interactions. It is unclear how to adapt LMs to new application intents without degrading the performance on existing applications. In this paper, we propose a solution to (a) estimate n-gram counts directly from the hand-written grammar for training LMs and (b) use constrained optimization to optimize the system parameters for future use cases, while not degrading the performance on past usage. We evaluated our approach on new applications intents for a personal assistant system and find that the adaptation improves the word error rate by up to 15% on new applications even when there is no adaptation data available for an application.