# Broadcasted Residual Learning for Efficient Keyword Spotting

arXiv: https://arxiv.org/abs/2106.04140 · HTML: https://mlsanity.com/paper/2106.04140/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 29 references that are in the map.
- First posted on arXiv 8 Jun 2021.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices](https://mlsanity.com/paper/1707.01083/) (2017)
- [Learning Transferable Architectures for Scalable Image Recognition](https://mlsanity.com/paper/1707.07012/) (2017)
- [Regularized Evolution for Image Classifier Architecture Search](https://mlsanity.com/paper/1802.01548/) (2018)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [Deep Residual Learning for Small-Footprint Keyword Spotting](https://mlsanity.com/paper/1710.10361/) (2017)
- [Temporal Convolution for Real-time Keyword Spotting on Mobile Devices](https://mlsanity.com/paper/1904.03814/) (2019)
- [Streaming keyword spotting on mobile devices](https://mlsanity.com/paper/2005.06720/) (2020)
- [A neural attention model for speech command recognition](https://mlsanity.com/paper/1808.08929/) (2018)
- [MatchboxNet: 1D Time-Channel Separable Convolutional Neural Network Architecture for Speech Commands Recognition](https://mlsanity.com/paper/2004.08531/) (2020)
- [Training Keyword Spotters with Limited and Synthesized Speech Data](https://mlsanity.com/paper/2002.01322/) (2020)
- [Query-by-example on-device keyword spotting](https://mlsanity.com/paper/1910.05171/) (2019)
- [Depthwise Separable Convolutional ResNet with Squeeze-and-Excitation Blocks for Small-footprint Keyword Spotting](https://mlsanity.com/paper/2004.12200/) (2020)
- [Small-Footprint Keyword Spotting with Multi-Scale Temporal Convolution](https://mlsanity.com/paper/2010.09960/) (2020)
- [SubSpectral Normalization for Neural Audio Data Processing](https://mlsanity.com/paper/2103.13620/) (2021)
- [URNet : User-Resizable Residual Networks with Conditional Gating Module](https://mlsanity.com/paper/1901.04687/) (2019)
- [Orthogonality Constrained Multi-Head Attention For Keyword Spotting](https://mlsanity.com/paper/1910.04500/) (2019)
- [Broadcasting Convolutional Network for Visual Relational Reasoning](https://mlsanity.com/paper/1712.02517/) (2017)

## Built on by (most cited first, 30 of 48)

- [Deep Spoken Keyword Spotting: An Overview](https://mlsanity.com/paper/2111.10592/) (2021)
- [Bytes Are All You Need: Transformers Operating Directly On File Bytes](https://mlsanity.com/paper/2306.00238/) (2023)
- [Domain Generalization with Relaxed Instance Frequency-wise Normalization for Multi-device Acoustic Scene Classification](https://mlsanity.com/paper/2206.12513/) (2022)
- [Leveraging Large Language Models for Exploiting ASR Uncertainty](https://mlsanity.com/paper/2309.04842/) (2023)
- [QTI Submission to DCASE 2021: residual normalization for device-imbalanced acoustic scene classification with efficient design](https://mlsanity.com/paper/2206.13909/) (2022)
- [Multi-task Voice Activated Framework using Self-supervised Learning](https://mlsanity.com/paper/2110.01077/) (2021)
- [Rainbow Keywords: Efficient Incremental Learning for Online Spoken Keyword Spotting](https://mlsanity.com/paper/2203.16361/) (2022)
- [Dummy Prototypical Networks for Few-Shot Open-Set Keyword Spotting](https://mlsanity.com/paper/2206.13691/) (2022)
- [Personalized Keyword Spotting through Multi-task Learning](https://mlsanity.com/paper/2206.13708/) (2022)
- [Domain Generalization on Efficient Acoustic Scene Classification using Residual Normalization](https://mlsanity.com/paper/2111.06531/) (2021)
- [BiFSMN: Binary Neural Network for Keyword Spotting](https://mlsanity.com/paper/2202.06483/) (2022)
- [Phase-driven Domain Generalizable Learning for Nonstationary Time Series](https://mlsanity.com/paper/2402.05960/) (2024)
- [Continual Learning For On-Device Environmental Sound Classification](https://mlsanity.com/paper/2207.07429/) (2022)
- [Learning Temporal Resolution in Spectrogram for Audio Classification](https://mlsanity.com/paper/2210.01719/) (2022)
- [LiCo-Net: Linearized Convolution Network for Hardware-efficient Keyword Spotting](https://mlsanity.com/paper/2211.04635/) (2022)
- [BiFSMNv2: Pushing Binary Neural Networks for Keyword Spotting to Real-Network Performance](https://mlsanity.com/paper/2211.06987/) (2022)
- [AnalyticKWS: Towards Exemplar-Free Analytic Class Incremental Learning for Small-footprint Keyword Spotting](https://mlsanity.com/paper/2505.11817/) (2025)
- [When the Inference Meets the Explicitness or Why Multimodality Can Make Us Forget About the Perfect Predictor](https://mlsanity.com/paper/2602.18850/) (2026)
- [HEiMDaL: Highly Efficient Method for Detection and Localization of wake-words](https://mlsanity.com/paper/2210.15425/) (2022)
- [Filterbank Learning for Noise-Robust Small-Footprint Keyword Spotting](https://mlsanity.com/paper/2211.10565/) (2022)
- [Improving vision-inspired keyword spotting using dynamic module skipping in streaming conformer encoder](https://mlsanity.com/paper/2309.00140/) (2023)
- [Count What You Want: Exemplar Identification and Few-shot Counting of Human Actions in the Wild](https://mlsanity.com/paper/2312.17330/) (2023)
- [To Wake-up or Not to Wake-up: Reducing Keyword False Alarm by Successive Refinement](https://mlsanity.com/paper/2304.03416/) (2023)
- [Hardware-accelerated graph neural networks: an alternative approach for neuromorphic event-based audio classification and keyword spotting on SoC FPGA](https://mlsanity.com/paper/2602.16442/) (2026)
- [I see what you hear: a vision-inspired method to localize words](https://mlsanity.com/paper/2210.13567/) (2022)
- [Robust Wake-Up Word Detection by Two-stage Multi-resolution Ensembles](https://mlsanity.com/paper/2310.11379/) (2023)
- [Sparse Binarization for Fast Keyword Spotting](https://mlsanity.com/paper/2406.06634/) (2024)
- [A Temporal-oriented Broadcast ResNet for COVID-19 Detection](https://mlsanity.com/paper/2203.17012/) (2022)
- [Scalable Weight Reparametrization for Efficient Transfer Learning](https://mlsanity.com/paper/2302.13435/) (2023)
- [Improving Small Footprint Few-shot Keyword Spotting with Supervision on Auxiliary Data](https://mlsanity.com/paper/2309.00647/) (2023)

## Abstract (arXiv)

Keyword spotting is an important research field because it plays a key role in device wake-up and user interaction on smart devices. However, it is challenging to minimize errors while operating efficiently in devices with limited resources such as mobile phones. We present a broadcasted residual learning method to achieve high accuracy with small model size and computational load. Our method configures most of the residual functions as 1D temporal convolution while still allows 2D convolution together using a broadcasted-residual connection that expands temporal output to frequency-temporal dimension. This residual mapping enables the network to effectively represent useful audio features with much less computation than conventional convolutional neural networks. We also propose a novel network architecture, Broadcasting-residual network (BC-ResNet), based on broadcasted residual learning and describe how to scale up the model according to the target device's resources. BC-ResNets achieve state-of-the-art 98.0% and 98.7% top-1 accuracy on Google speech command datasets v1 and v2, respectively, and consistently outperform previous approaches, using fewer computations and parameters. Code is available at https://github.com/Qualcomm-AI-research/bcresnet.