mlsanityData to 14 Oct 2026

PCOV-KWS: Multi-task Learning for Personalized Customizable Open Vocabulary Keyword Spotting

arXiv 2603.18023 ↗First posted 5 Mar 2026eess.AS cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

As advancements in technologies like Internet of Things (IoT), Automatic Speech Recognition (ASR), Speaker Verification (SV), and Text-to-Speech (TTS) lead to increased usage of intelligent voice assistants, the demand for privacy and personalization has escalated. In this paper, we introduce a multi-task learning framework for personalized, customizable open-vocabulary Keyword Spotting (PCOV-KWS). This framework employs a lightweight network to simultaneously perform Keyword Spotting (KWS) and SV to address personalized KWS requirements. We have integrated a training criterion distinct from softmax-based loss, transforming multi-class classification into multiple binary classifications, which eliminates inter-category competition, while an optimization strategy for multi-task loss weighting is employed during training. We evaluated our PCOV-KWS system in multiple datasets, demonstrating that it outperforms the baselines in evaluation results, while also requiring fewer parameters and lower computational resources.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 28 of its 39 references that are in the map. First posted on arXiv 5 Mar 2026.

Builds on

28 of 39 references.

References · 28

  1. 2022A ConvNet for the 2020scited by 3,579
  2. 2020Gradient Surgery for Multi-Task Learningcited by 1,256
  3. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 752
  4. 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 712
  5. 2020In defence of metric learning for speaker recognitioncited by 126
  6. 2017Deep Residual Learning for Small-Footprint Keyword Spottingcited by 68
  7. 2019Temporal Convolution for Real-time Keyword Spotting on Mobile Devicescited by 49
  8. 2021Broadcasted Residual Learning for Efficient Keyword Spottingcited by 48
  9. 2020Streaming keyword spotting on mobile devicescited by 47
  10. 2018A neural attention model for speech command recognitioncited by 45
  11. 2021SphereFace2: Binary Classification is All You Need for Deep Face Recognitioncited by 42
  12. 2020MatchboxNet: 1D Time-Channel Separable Convolutional Neural Network Architecture for Speech Commands Recognitioncited by 36
16 more
  1. 2015Deep convolutional acoustic word embeddings using word-pair side informationcited by 27
  2. 2021Query-by-Example Keyword Spotting system using Multi-head Attention and Softtriple Losscited by 25
  3. 2017Query-by-Example Search with Discriminative Neural Acoustic Word Embeddingscited by 22
  4. 2022Learning Audio-Text Agreement for Open-vocabulary Keyword Spottingcited by 20
  5. 2020Noisy Differentiable Architecture Searchcited by 16
  6. 2018DONUT: CTC-based Query-by-Example Keyword Spottingcited by 15
  7. 2020Small-Footprint Keyword Spotting with Multi-Scale Temporal Convolutioncited by 15
  8. 2023Matching Latent Encoding for Audio-Text based Keyword Spottingcited by 15
  9. 2023PhonMatchNet: Phoneme-Guided Zero-Shot Keyword Spotting for User-Defined Keywordscited by 12
  10. 2020AutoKWS: Keyword Spotting with Differentiable Architecture Searchcited by 10
  11. 2020Multi-Task Network for Noise-Robust Keyword Spotting and Speaker Verification using CTC-based Soft VAD and Global Query Attentioncited by 9
  12. 2022Personalized Keyword Spotting through Multi-task Learningcited by 7
  13. 2022Metric Learning for User-defined Keyword Spottingcited by 7
  14. 2019Additional Shared Decoder on Siamese Multi-view Encoders for Learning Acoustic Word Embeddingscited by 6
  15. 2023Multilingual Query-by-Example Keyword Spotting with Metric Learning and Phoneme-to-Embedding Mappingcited by 3
  16. 2024Contrastive Learning With Audio Discrimination For Customizable Keyword Spotting In Continuous Speechcited by 2

Built on by

Not cited yet.