mlsanityData to 7 Oct 2026

Personalized Keyword Spotting through Multi-task Learning

arXiv 2206.13708 ↗First posted 28 Jun 2022cs.SD cs.LG eess.AScited by 7

Not on the map. Why?

Abstract (arXiv)

Keyword spotting (KWS) plays an essential role in enabling speech-based user interaction on smart devices, and conventional KWS (C-KWS) approaches have concentrated on detecting user-agnostic pre-defined keywords. However, in practice, most user interactions come from target users enrolled in the device which motivates to construct personalized keyword spotting. We design two personalized KWS tasks; (1) Target user Biased KWS (TB-KWS) and (2) Target user Only KWS (TO-KWS). To solve the tasks, we propose personalized keyword spotting through multi-task learning (PK-MTL) that consists of multi-task learning and task-adaptation. First, we introduce applying multi-task learning on keyword spotting and speaker verification to leverage user information to the keyword spotting system. Next, we design task-specific scoring functions to adapt to the personalized KWS tasks thoroughly. We evaluate our framework on conventional and personalized scenarios, and the results show that PK-MTL can dramatically reduce the false alarm rate, especially in various practical scenarios.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 19 of its 27 references that are in the map. First posted on arXiv 28 Jun 2022.

Builds on

19 of 27 references.

References · 19

  1. 2017Squeeze-and-Excitation Networkscited by 4,763
  2. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 741
  3. 2017SphereFace: Deep Hypersphere Embedding for Face Recognitioncited by 733
  4. 2020In defence of metric learning for speaker recognitioncited by 123
  5. 2019Prototype Rectification for Few-Shot Learningcited by 96
  6. 2017Deep Residual Learning for Small-Footprint Keyword Spottingcited by 68
  7. 2019Temporal Convolution for Real-time Keyword Spotting on Mobile Devicescited by 48
  8. 2020Streaming keyword spotting on mobile devicescited by 47
  9. 2020Reparameterizing Convolutions for Incremental Multi-Task Learning without Task Interferencecited by 47
  10. 2021Broadcasted Residual Learning for Efficient Keyword Spottingcited by 47
  11. 2021Query-by-Example Keyword Spotting system using Multi-head Attention and Softtriple Losscited by 25
  12. 2019Small-Footprint Keyword Spotting on Raw Audio Data with Sinc-Convolutionscited by 20
7 more
  1. 2019Query-by-example on-device keyword spottingcited by 19
  2. 2017A Cascade Architecture for Keyword Spotting on Mobile Devicescited by 15
  3. 2020Depthwise Separable Convolutional ResNet with Squeeze-and-Excitation Blocks for Small-footprint Keyword Spottingcited by 15
  4. 2020AutoKWS: Keyword Spotting with Differentiable Architecture Searchcited by 10
  5. 2020Multi-Task Network for Noise-Robust Keyword Spotting and Speaker Verification using CTC-based Soft VAD and Global Query Attentioncited by 9
  6. 2018Learning Acoustic Word Embeddings with Temporal Context for Query-by-Example Speech Searchcited by 7
  7. 2019An End-to-End Text-independent Speaker Verification Framework with a Keyword Adversarial Networkcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 1; 2025: 0; 2026: 2.

7 papers.

  1. 2023Few-Shot Open-Set Learning for On-Device Customization of KeyWord Spotting Systemscited by 9
  2. 2024Self-Learning for Personalized Keyword Spotting on Ultra-Low-Power Audio Sensorscited by 7
  3. 2023Latent Phrase Matching for Dysarthric Speechcited by 2
  4. 2023Personalizing Keyword Spotting with Speaker Informationcited by 1
  5. 2023Improving Small Footprint Few-shot Keyword Spotting with Supervision on Auxiliary Datacited by 0
  6. 2026OnDA: On-device Channel Pruning for Efficient Personalized Keyword Spottingcited by 0
  7. 2026PCOV-KWS: Multi-task Learning for Personalized Customizable Open Vocabulary Keyword Spottingcited by 0