# RepCNN: Micro-sized, Mighty Models for Wakeword Detection

arXiv: https://arxiv.org/abs/2406.02652 · HTML: https://mlsanity.com/paper/2406.02652/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 28 references that are in the map.
- First posted on arXiv 4 Jun 2024.

## Builds on (most cited first, 20 of 20)

- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design](https://mlsanity.com/paper/1807.11164/) (2018)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [RepVGG: Making VGG-style ConvNets Great Again](https://mlsanity.com/paper/2101.03697/) (2021)
- [RepViT: Revisiting Mobile CNN From ViT Perspective](https://mlsanity.com/paper/2307.09283/) (2023)
- [MobileOne: An Improved One millisecond Mobile Backbone](https://mlsanity.com/paper/2206.04040/) (2022)
- [Deep Residual Learning for Small-Footprint Keyword Spotting](https://mlsanity.com/paper/1710.10361/) (2017)
- [Temporal Convolution for Real-time Keyword Spotting on Mobile Devices](https://mlsanity.com/paper/1904.03814/) (2019)
- [Broadcasted Residual Learning for Efficient Keyword Spotting](https://mlsanity.com/paper/2106.04140/) (2021)
- [MatchboxNet: 1D Time-Channel Separable Convolutional Neural Network Architecture for Speech Commands Recognition](https://mlsanity.com/paper/2004.08531/) (2020)
- [End-to-End Streaming Keyword Spotting](https://mlsanity.com/paper/1812.02802/) (2018)
- [Small-Footprint Keyword Spotting with Multi-Scale Temporal Convolution](https://mlsanity.com/paper/2010.09960/) (2020)
- [RepGhost: A Hardware-Efficient Ghost Module via Re-parameterization](https://mlsanity.com/paper/2211.06088/) (2022)
- [Optimize what matters: Training DNN-HMM Keyword Spotting Model Using End Metric](https://mlsanity.com/paper/2011.01151/) (2020)
- [Stacked 1D convolutional networks for end-to-end small footprint voice trigger detection](https://mlsanity.com/paper/2008.03405/) (2020)
- [Online Keyword Spotting with a Character-Level Recurrent Neural Network](https://mlsanity.com/paper/1512.08903/) (2015)
- [An End-to-End Architecture for Keyword Spotting and Voice Activity Detection](https://mlsanity.com/paper/1611.09405/) (2016)
- [HEiMDaL: Highly Efficient Method for Detection and Localization of wake-words](https://mlsanity.com/paper/2210.15425/) (2022)

## Abstract (arXiv)

Always-on machine learning models require a very low memory and compute footprint. Their restricted parameter count limits the model's capacity to learn, and the effectiveness of the usual training algorithms to find the best parameters. Here we show that a small convolutional model can be better trained by first refactoring its computation into a larger redundant multi-branched architecture. Then, for inference, we algebraically re-parameterize the trained model into the single-branched form with fewer parameters for a lower memory footprint and compute cost. Using this technique, we show that our always-on wake-word detector model, RepCNN, provides a good trade-off between latency and accuracy during inference. RepCNN re-parameterized models are 43% more accurate than a uni-branch convolutional model while having the same runtime. RepCNN also meets the accuracy of complex architectures like BC-ResNet, while having 2x lesser peak memory usage and 10x faster runtime.