# Multi-Task Network for Noise-Robust Keyword Spotting and Speaker Verification using CTC-based Soft VAD and Global Query Attention

arXiv: https://arxiv.org/abs/2005.03867 · HTML: https://mlsanity.com/paper/2005.03867/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 May 2020.

## Built on by (most cited first, 9 of 9)

- [Deep Spoken Keyword Spotting: An Overview](https://mlsanity.com/paper/2111.10592/) (2021)
- [On-Device Domain Learning for Keyword Spotting on Low-Power Extreme Edge Embedded Systems](https://mlsanity.com/paper/2403.10549/) (2024)
- [Personalized Keyword Spotting through Multi-task Learning](https://mlsanity.com/paper/2206.13708/) (2022)
- [Learning Decoupling Features Through Orthogonality Regularization](https://mlsanity.com/paper/2203.16772/) (2022)
- [Adaptive Noise Resilient Keyword Spotting Using One-Shot Learning](https://mlsanity.com/paper/2505.09304/) (2025)
- [A Study of Designing Compact Audio-Visual Wake Word Spotting System Based on Iterative Fine-Tuning in Neural Network Pruning](https://mlsanity.com/paper/2202.08509/) (2022)
- [A Multi-tasking Model of Speaker-Keyword Classification for Keeping Human in the Loop of Drone-assisted Inspection](https://mlsanity.com/paper/2207.04027/) (2022)
- [Advancing Airport Tower Command Recognition: Integrating Squeeze-and-Excitation and Broadcasted Residual Learning](https://mlsanity.com/paper/2406.18313/) (2024)
- [PCOV-KWS: Multi-task Learning for Personalized Customizable Open Vocabulary Keyword Spotting](https://mlsanity.com/paper/2603.18023/) (2026)

## Abstract (arXiv)

Keyword spotting (KWS) and speaker verification (SV) have been studied independently although it is known that acoustic and speaker domains are complementary. In this paper, we propose a multi-task network that performs KWS and SV simultaneously to fully utilize the interrelated domain information. The multi-task network tightly combines sub-networks aiming at performance improvement in challenging conditions such as noisy environments, open-vocabulary KWS, and short-duration SV, by introducing novel techniques of connectionist temporal classification (CTC)-based soft voice activity detection (VAD) and global query attention. Frame-level acoustic and speaker information is integrated with phonetically originated weights so that forms a word-level global representation. Then it is used for the aggregation of feature vectors to generate discriminative embeddings. Our proposed approach shows 4.06% and 26.71% relative improvements in equal error rate (EER) compared to the baselines for both tasks. We also present a visualization example and results of ablation experiments.