# Fixed-point quantization aware training for on-device keyword-spotting

arXiv: https://arxiv.org/abs/2303.02284 · HTML: https://mlsanity.com/paper/2303.02284/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 30 references that are in the map.
- First posted on arXiv 4 Mar 2023.

## Builds on (most cited first, 14 of 14)

- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [BinaryConnect: Training Deep Neural Networks with binary weights during propagations](https://mlsanity.com/paper/1511.00363/) (2015)
- [FINN: A Framework for Fast, Scalable Binarized Neural Network Inference](https://mlsanity.com/paper/1612.07119/) (2016)
- [TernaryBERT: Distillation-aware Ultra-low Bit BERT](https://mlsanity.com/paper/2009.12812/) (2020)
- [Fixed Point Quantization of Deep Convolutional Networks](https://mlsanity.com/paper/1511.06393/) (2015)
- [Training Quantized Nets: A Deeper Understanding](https://mlsanity.com/paper/1706.02379/) (2017)
- [Regularizing Activation Distribution for Training Binarized Deep Networks](https://mlsanity.com/paper/1904.02823/) (2019)
- [CascadeCNN: Pushing the Performance Limits of Quantisation in Convolutional Neural Networks](https://mlsanity.com/paper/1807.05053/) (2018)
- [Compression of Acoustic Event Detection Models with Low-rank Matrix Factorization and Quantization Training](https://mlsanity.com/paper/1905.00855/) (2019)
- [Sub 8-Bit Quantization of Streaming Keyword Spotting Models for Embedded Chipsets](https://mlsanity.com/paper/2207.06920/) (2022)
- [Weight Reparametrization for Budget-Aware Network Pruning](https://mlsanity.com/paper/2107.03909/) (2021)

## Built on by (most cited first, 2 of 2)

- [Robust Wake-Up Word Detection by Two-stage Multi-resolution Ensembles](https://mlsanity.com/paper/2310.11379/) (2023)
- [Hierarchical Spatial Algorithms for High-Resolution Image Quantization and Feature Extraction](https://mlsanity.com/paper/2510.08449/) (2025)

## Abstract (arXiv)

Fixed-point (FXP) inference has proven suitable for embedded devices with limited computational resources, and yet model training is continually performed in floating-point (FLP). FXP training has not been fully explored and the non-trivial conversion from FLP to FXP presents unavoidable performance drop. We propose a novel method to train and obtain FXP convolutional keyword-spotting (KWS) models. We combine our methodology with two quantization-aware-training (QAT) techniques - squashed weight distribution and absolute cosine regularization for model parameters, and propose techniques for extending QAT over transient variables, otherwise neglected by previous paradigms. Experimental results on the Google Speech Commands v2 dataset show that we can reduce model precision up to 4-bit with no loss in accuracy. Furthermore, on an in-house KWS dataset, we show that our 8-bit FXP-QAT models have a 4-6% improvement in relative false discovery rate at fixed false reject rate compared to full precision FLP models. During inference we argue that FXP-QAT eliminates q-format normalization and enables the use of low-bit accumulators while maximizing SIMD throughput to reduce user perceived latency. We demonstrate that we can reduce execution time by 68% without compromising KWS model's predictive performance or requiring model architectural changes. Our work provides novel findings that aid future research in this area and enable accurate and efficient models.