# Max-Pooling Loss Training of Long Short-Term Memory Networks for Small-Footprint Keyword Spotting

arXiv: https://arxiv.org/abs/1705.02411 · HTML: https://mlsanity.com/paper/1705.02411/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 May 2017.

## Built on by (most cited first, 15 of 15)

- [Deep Spoken Keyword Spotting: An Overview](https://mlsanity.com/paper/2111.10592/) (2021)
- [Learning Efficient Representations for Keyword Spotting with Triplet Loss](https://mlsanity.com/paper/2101.04792/) (2021)
- [Visual Keyword Spotting with Attention](https://mlsanity.com/paper/2110.15957/) (2021)
- [Spike-inspired Rank Coding for Fast and Accurate Recurrent Neural Networks](https://mlsanity.com/paper/2110.02865/) (2021)
- [Keyword Spotting System and Evaluation of Pruning and Quantization Methods on Low-power Edge Microcontrollers](https://mlsanity.com/paper/2208.02765/) (2022)
- [LiCo-Net: Linearized Convolution Network for Hardware-efficient Keyword Spotting](https://mlsanity.com/paper/2211.04635/) (2022)
- [Enhancing Voice Wake-Up for Dysarthria: Mandarin Dysarthria Speech Corpus Release and Customized System Design](https://mlsanity.com/paper/2406.10304/) (2024)
- [End-to-end Keyword Spotting using Neural Architecture Search and Quantization](https://mlsanity.com/paper/2104.06666/) (2021)
- [Exploring Sequence-to-Sequence Transformer-Transducer Models for Keyword Spotting](https://mlsanity.com/paper/2211.06478/) (2022)
- [To Wake-up or Not to Wake-up: Reducing Keyword False Alarm by Successive Refinement](https://mlsanity.com/paper/2304.03416/) (2023)
- [Improving Voice Trigger Detection with Metric Learning](https://mlsanity.com/paper/2204.02455/) (2022)
- [Speech Augmentation Based Unsupervised Learning for Keyword Spotting](https://mlsanity.com/paper/2205.14329/) (2022)
- [Latency Control for Keyword Spotting](https://mlsanity.com/paper/2206.07261/) (2022)
- [Personalizing Keyword Spotting with Speaker Information](https://mlsanity.com/paper/2311.03419/) (2023)
- [Geometric Knowledge-Assisted Federated Dual Knowledge Distillation Approach Towards Remote Sensing Satellite Imagery](https://mlsanity.com/paper/2603.07774/) (2026)

## Abstract (arXiv)

We propose a max-pooling based loss function for training Long Short-Term Memory (LSTM) networks for small-footprint keyword spotting (KWS), with low CPU, memory, and latency requirements. The max-pooling loss training can be further guided by initializing with a cross-entropy loss trained network. A posterior smoothing based evaluation approach is employed to measure keyword spotting performance. Our experimental results show that LSTM models trained using cross-entropy loss or max-pooling loss outperform a cross-entropy loss trained baseline feed-forward Deep Neural Network (DNN). In addition, max-pooling loss trained LSTM with randomly initialized network performs better compared to cross-entropy loss trained LSTM. Finally, the max-pooling loss trained LSTM initialized with a cross-entropy pre-trained network shows the best performance, which yields 67.6% relative reduction compared to baseline feed-forward DNN in Area Under the Curve (AUC) measure.