# Streaming Keyword Spotting Boosted by Cross-layer Discrimination Consistency

arXiv: https://arxiv.org/abs/2412.12635 · HTML: https://mlsanity.com/paper/2412.12635/

## Key facts

- Cited by 1 papers in the map (data through 19 Oct 2026).
- Placed in Speech recognition models and training, one of 11 directions in Speech and audio language models.
- Builds on 20 of its 36 references that are in the map.
- First posted on arXiv 17 Dec 2024.

## Placed in

- Speech and audio language models / [Speech recognition models and training](https://mlsanity.com/speech/speech-recognition-models-and-training/)

## Builds on (most cited first, 20 of 20)

- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [WHAM!: Extending Speech Separation to Noisy Environments](https://mlsanity.com/paper/1907.01160/) (2019)
- [Intermediate Loss Regularization for CTC-based Speech Recognition](https://mlsanity.com/paper/2102.03216/) (2021)
- [Relaxing the Conditional Independence Assumption of CTC-based ASR by Conditioning on Intermediate Predictions](https://mlsanity.com/paper/2104.02724/) (2021)
- [Deep Spoken Keyword Spotting: An Overview](https://mlsanity.com/paper/2111.10592/) (2021)
- [Deep-FSMN for Large Vocabulary Continuous Speech Recognition](https://mlsanity.com/paper/1803.05030/) (2018)
- [Improving Massively Multilingual ASR With Auxiliary CTC Objectives](https://mlsanity.com/paper/2302.12829/) (2023)
- [First-Pass Large Vocabulary Continuous Speech Recognition using Bi-Directional Recurrent DNNs](https://mlsanity.com/paper/1408.2873/) (2014)
- [Efficient keyword spotting using dilated convolutions and gating](https://mlsanity.com/paper/1811.07684/) (2018)
- [End-to-End Streaming Keyword Spotting](https://mlsanity.com/paper/1812.02802/) (2018)
- [Hierarchical Conditional End-to-End ASR with CTC and Multi-Granular Subword Units](https://mlsanity.com/paper/2110.04109/) (2021)
- [Small-Footprint Open-Vocabulary Keyword Spotting with Quantized LSTM Networks](https://mlsanity.com/paper/2002.10851/) (2020)
- [Multi-Task Network for Noise-Robust Keyword Spotting and Speaker Verification using CTC-based Soft VAD and Global Query Attention](https://mlsanity.com/paper/2005.03867/) (2020)
- [Wake Word Detection with Streaming Transformers](https://mlsanity.com/paper/2102.04488/) (2021)
- [Multilingual Audio-Visual Speech Recognition with Hybrid CTC/RNN-T Fast Conformer](https://mlsanity.com/paper/2405.12983/) (2024)
- [WeKws: A production first small-footprint end-to-end Keyword Spotting Toolkit](https://mlsanity.com/paper/2210.16743/) (2022)
- [Contrastive Learning With Audio Discrimination For Customizable Keyword Spotting In Continuous Speech](https://mlsanity.com/paper/2401.06485/) (2024)
- [Text-aware Speech Separation for Multi-talker Keyword Spotting](https://mlsanity.com/paper/2406.12447/) (2024)
- [TDT-KWS: Fast And Accurate Keyword Spotting Using Token-and-duration Transducer](https://mlsanity.com/paper/2403.13332/) (2024)
- [U2-KWS: Unified Two-pass Open-vocabulary Keyword Spotting with Keyword Bias](https://mlsanity.com/paper/2312.09760/) (2023)

## Built on by (most cited first, 1 of 1)

- [Neural Directed Speech Enhancement with Dual Microphone Array in High Noise Scenario](https://mlsanity.com/paper/2412.18141/) (2024)

## Abstract (arXiv)

Connectionist Temporal Classification (CTC), a non-autoregressive training criterion, is widely used in online keyword spotting (KWS). However, existing CTC-based KWS decoding strategies either rely on Automatic Speech Recognition (ASR), which performs suboptimally due to its broad search over the acoustic space without keyword-specific optimization, or on KWS-specific decoding graphs, which are complex to implement and maintain. In this work, we propose a streaming decoding algorithm enhanced by Cross-layer Discrimination Consistency (CDC), tailored for CTC-based KWS. Specifically, we introduce a streamlined yet effective decoding algorithm capable of detecting the start of the keyword at any arbitrary position. Furthermore, we leverage discrimination consistency information across layers to better differentiate between positive and false alarm samples. Our experiments on both clean and noisy Hey Snips datasets show that the proposed streaming decoding strategy outperforms ASR-based and graph-based KWS baselines. The CDC-boosted decoding further improves performance, yielding an average absolute recall improvement of 6.8% and a 46.3% relative reduction in the miss rate compared to the graph-based KWS baseline, with a very low false alarm rate of 0.05 per hour.