# Optimize what matters: Training DNN-HMM Keyword Spotting Model Using End Metric

arXiv: https://arxiv.org/abs/2011.01151 · HTML: https://mlsanity.com/paper/2011.01151/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Nov 2020.

## Built on by (most cited first, 9 of 9)

- [Synt++: Utilizing Imperfect Synthetic Data to Improve Speech Recognition](https://mlsanity.com/paper/2110.11479/) (2021)
- [Streaming Transformer for Hardware Efficient Voice Trigger Detection and False Trigger Mitigation](https://mlsanity.com/paper/2105.06598/) (2021)
- [HEiMDaL: Highly Efficient Method for Detection and Localization of wake-words](https://mlsanity.com/paper/2210.15425/) (2022)
- [I see what you hear: a vision-inspired method to localize words](https://mlsanity.com/paper/2210.13567/) (2022)
- [Streaming Anchor Loss: Augmenting Supervision with Temporal Significance](https://mlsanity.com/paper/2310.05886/) (2023)
- [A Study of Designing Compact Audio-Visual Wake Word Spotting System Based on Iterative Fine-Tuning in Neural Network Pruning](https://mlsanity.com/paper/2202.08509/) (2022)
- [Marginalized Beam Search Algorithms for Hierarchical HMMs](https://mlsanity.com/paper/2305.11752/) (2023)
- [Conditional Online Learning for Keyword Spotting](https://mlsanity.com/paper/2305.13332/) (2023)
- [RepCNN: Micro-sized, Mighty Models for Wakeword Detection](https://mlsanity.com/paper/2406.02652/) (2024)

## Abstract (arXiv)

Deep Neural Network--Hidden Markov Model (DNN-HMM) based methods have been successfully used for many always-on keyword spotting algorithms that detect a wake word to trigger a device. The DNN predicts the state probabilities of a given speech frame, while HMM decoder combines the DNN predictions of multiple speech frames to compute the keyword detection score. The DNN, in prior methods, is trained independent of the HMM parameters to minimize the cross-entropy loss between the predicted and the ground-truth state probabilities. The mis-match between the DNN training loss (cross-entropy) and the end metric (detection score) is the main source of sub-optimal performance for the keyword spotting task. We address this loss-metric mismatch with a novel end-to-end training strategy that learns the DNN parameters by optimizing for the detection score. To this end, we make the HMM decoder (dynamic programming) differentiable and back-propagate through it to maximize the score for the keyword and minimize the scores for non-keyword speech segments. Our method does not require any change in the model architecture or the inference framework; therefore, there is no overhead in run-time memory or compute requirements. Moreover, we show significant reduction in false rejection rate (FRR) at the same false trigger experience (> 70% over independent DNN training).