Server-side Rescoring of Spoken Entity-centric Knowledge Queries for Virtual Assistants
Not on the map. Why?
Abstract (arXiv)
On-device Virtual Assistants (VAs) powered by Automatic Speech Recognition (ASR) require effective knowledge integration for the challenging entity-rich query recognition. In this paper, we conduct an empirical study of modeling strategies for server-side rescoring of spoken information domain queries using various categories of Language Models (LMs) (N-gram word LMs, sub-word neural LMs). We investigate the combination of on-device and server-side signals, and demonstrate significant WER improvements of 23%-35% on various entity-centric query subpopulations by integrating various server-side LMs compared to performing ASR on-device only. We also perform a comparison between LMs trained on domain data and a GPT-3 variant offered by OpenAI as a baseline. Furthermore, we also show that model fusion of multiple server-side LMs trained from scratch most effectively combines complementary strengths of each model and integrates knowledge learned from domain-specific data to a VA ASR system.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 20 of its 48 references that are in the map. First posted on arXiv 2 Nov 2023.
Builds on
20 of 48 references.
References · 20
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2017Deep reinforcement learning from human preferencescited by 3,846
- 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
- 2018Self-Attention with Relative Position Representationscited by 1,112
- 2019Language Modeling with Deep Transformerscited by 96
- 2022RescoreBERT: Discriminative Speech Recognition Rescoring with BERTcited by 34
- 2020Improving Tail Performance of a Deliberation E2E ASR Model Using a Large Text Corpuscited by 24
8 more
- 2019SNDCNN: Self-normalizing deep CNNs with scaled exponential linear units for speech recognitioncited by 13
- 2022Improving Rare Word Recognition with LM-aware MWER Trainingcited by 12
- 2019An Empirical Study of Efficient ASR Rescoring with Transformerscited by 11
- 2022Sentence-Select: Large-Scale Language Model Data Selection for Rare-Word Speech Recognitioncited by 10
- 2021On-device neural speech synthesiscited by 9
- 2020Predicting Entity Popularity to Improve Spoken Entity Recognition by Virtual Assistantscited by 8
- 2022Space-Efficient Representation of Entity-centric Query Language Modelscited by 7
- 2022ASR-Generated Text for Language Model Pre-training Applied to Speech Taskscited by 6
Built on by
Citing papers per year: 2024: 1; 2025: 1.
2 papers.