# End-to-end Keyword Spotting using Neural Architecture Search and Quantization

arXiv: https://arxiv.org/abs/2104.06666 · HTML: https://mlsanity.com/paper/2104.06666/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 35 references that are in the map.
- First posted on arXiv 14 Apr 2021.

## Builds on (most cited first, 24 of 24)

- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [MnasNet: Platform-Aware Neural Architecture Search for Mobile](https://mlsanity.com/paper/1807.11626/) (2018)
- [Efficient Neural Architecture Search via Parameter Sharing](https://mlsanity.com/paper/1802.03268/) (2018)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware](https://mlsanity.com/paper/1812.00332/) (2018)
- [Learned Step Size Quantization](https://mlsanity.com/paper/1902.08153/) (2019)
- [State-of-the-art Speech Recognition With Sequence-to-Sequence Models](https://mlsanity.com/paper/1712.01769/) (2017)
- [Hierarchical Representations for Efficient Architecture Search](https://mlsanity.com/paper/1711.00436/) (2017)
- [Speaker Recognition from Raw Waveform with SincNet](https://mlsanity.com/paper/1808.00158/) (2018)
- [Hello Edge: Keyword Spotting on Microcontrollers](https://mlsanity.com/paper/1711.07128/) (2017)
- [Trained Quantization Thresholds for Accurate and Efficient Fixed-Point Inference of Deep Neural Networks](https://mlsanity.com/paper/1903.08066/) (2019)
- [SpArSe: Sparse Architecture Search for CNNs on Resource-Constrained Microcontrollers](https://mlsanity.com/paper/1905.12107/) (2019)
- [TinyLSTMs: Efficient Neural Speech Enhancement for Hearing Aids](https://mlsanity.com/paper/2005.11138/) (2020)
- [Convolutional Recurrent Neural Networks for Small-Footprint Keyword Spotting](https://mlsanity.com/paper/1703.05390/) (2017)
- [Small-Footprint Keyword Spotting on Raw Audio Data with Sinc-Convolutions](https://mlsanity.com/paper/1911.02086/) (2019)
- [Mixed Precision DNNs: All you need is a good parametrization](https://mlsanity.com/paper/1905.11452/) (2019)
- [Max-Pooling Loss Training of Long Short-Term Memory Networks for Small-Footprint Keyword Spotting](https://mlsanity.com/paper/1705.02411/) (2017)
- [Neural Architecture Search For Keyword Spotting](https://mlsanity.com/paper/2009.00165/) (2020)
- [AutoKWS: Keyword Spotting with Differentiable Architecture Search](https://mlsanity.com/paper/2009.03658/) (2020)
- [Deep Convolutional Neural Network Inference with Floating-point Weights and Fixed-point Activations](https://mlsanity.com/paper/1703.03073/) (2017)
- [Honk: A PyTorch Reimplementation of Convolutional Neural Networks for Keyword Spotting](https://mlsanity.com/paper/1710.06554/) (2017)
- [Resource-efficient DNNs for Keyword Spotting using Neural Architecture Search and Quantization](https://mlsanity.com/paper/2012.10138/) (2020)

## Built on by (most cited first, 2 of 2)

- [Filterbank Learning for Noise-Robust Small-Footprint Keyword Spotting](https://mlsanity.com/paper/2211.10565/) (2022)
- [A Multi-tasking Model of Speaker-Keyword Classification for Keeping Human in the Loop of Drone-assisted Inspection](https://mlsanity.com/paper/2207.04027/) (2022)

## Abstract (arXiv)

This paper introduces neural architecture search (NAS) for the automatic discovery of end-to-end keyword spotting (KWS) models in limited resource environments. We employ a differentiable NAS approach to optimize the structure of convolutional neural networks (CNNs) operating on raw audio waveforms. After a suitable KWS model is found with NAS, we conduct quantization of weights and activations to reduce the memory footprint. We conduct extensive experiments on the Google speech commands dataset. In particular, we compare our end-to-end approach to mel-frequency cepstral coefficient (MFCC) based systems. For quantization, we compare fixed bit-width quantization and trained bit-width quantization. Using NAS only, we were able to obtain a highly efficient model with an accuracy of 95.55% using 75.7k parameters and 13.6M operations. Using trained bit-width quantization, the same model achieves a test accuracy of 93.76% while using on average only 2.91 bits per activation and 2.51 bits per weight.