Temporal Convolution for Real-time Keyword Spotting on Mobile Devices
Not on the map. Why?
Abstract (arXiv)
Keyword spotting (KWS) plays a critical role in enabling speech-based user interactions on smart devices. Recent developments in the field of deep learning have led to wide adoption of convolutional neural networks (CNNs) in KWS systems due to their exceptional accuracy and robustness. The main challenge faced by KWS systems is the trade-off between high accuracy and low latency. Unfortunately, there has been little quantitative analysis of the actual latency of KWS models on mobile devices. This is especially concerning since conventional convolution-based KWS approaches are known to require a large number of operations to attain an adequate level of performance. In this paper, we propose a temporal convolution for real-time KWS on mobile devices. Unlike most of the 2D convolution-based KWS approaches that require a deep architecture to fully capture both low- and high-frequency domains, we exploit temporal convolutions with a compact ResNet architecture. In Google Speech Command Dataset, we achieve more than 385x speedup on Google Pixel 1 and surpass the accuracy compared to the state-of-the-art model. In addition, we release the implementation of the proposed and the baseline models including an end-to-end pipeline for training models and evaluating them on mobile devices.
Key facts. Cited by 49 papers in the map (data through 14 Oct 2026). First posted on arXiv 8 Apr 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 12; 2022: 12; 2023: 7; 2024: 8; 2025: 5; 2026: 5.
49 papers.
Sort:
By area:
- 2021Keyword Transformer: A Self-Attention Model for Keyword Spottingcited by 60
- 2021Broadcasted Residual Learning for Efficient Keyword Spottingcited by 48
- 2021Deep Spoken Keyword Spotting: An Overviewcited by 43
- 2022Position Prediction as an Effective Pretraining Strategycited by 24
- 2022Learning Audio-Text Agreement for Open-vocabulary Keyword Spottingcited by 20
- 2021Widening Access to Applied Machine Learning with TinyMLcited by 17
- 2021Temporal Knowledge Distillation for On-device Audio Classificationcited by 15
- 2023Lightweight Neural Architecture Search for Temporal Convolutional Networks at the Edgecited by 15
- 2022TC-SKNet with GridMask for Low-complexity Classification of Acoustic scenecited by 12
- 2021SubSpectral Normalization for Neural Audio Data Processingcited by 11
- 2021A Streaming End-to-End Framework For Spoken Language Understandingcited by 8
- 2022Progressive Continual Learning for Spoken Keyword Spottingcited by 8
37 more
- 2022Rainbow Keywords: Efficient Incremental Learning for Online Spoken Keyword Spottingcited by 8
- 2022Filler Word Detection and Classification: A Dataset and Benchmarkcited by 7
- 2022Personalized Keyword Spotting through Multi-task Learningcited by 7
- 2022Neural Architecture Search for Energy Efficient Always-on Audio Modelscited by 5
- 2021Streaming on-device detection of device directed speech from voice and touch-based invocationcited by 4
- 2022LiCo-Net: Linearized Convolution Network for Hardware-efficient Keyword Spottingcited by 4
- 2025AnalyticKWS: Towards Exemplar-Free Analytic Class Incremental Learning for Small-footprint Keyword Spottingcited by 4
- 2021Text Anchor Based Metric Learning for Small-footprint Keyword Spottingcited by 3
- 2021FakeWake: Understanding and Mitigating Fake Wake-up Words of Voice Assistantscited by 3
- 2024ED-sKWS: Early-Decision Spiking Neural Networks for Rapid,and Energy-Efficient Keyword Spottingcited by 3
- 2021Encoder-Decoder Neural Architecture Optimization for Keyword Spottingcited by 2
- 2023To Wake-up or Not to Wake-up: Reducing Keyword False Alarm by Successive Refinementcited by 2
- 2022Learning Decoupling Features Through Orthogonality Regularizationcited by 1
- 2022Improving Voice Trigger Detection with Metric Learningcited by 1
- 2023Exploring Representation Learning for Small-Footprint Keyword Spottingcited by 1
- 2023Federated Few-shot Learning for Cough Classification with Edge Devicescited by 1
- 2023VIC-KD: Variance-Invariance-Covariance Knowledge Distillation to Make Keyword Spotting More Robust Against Adversarial Attackscited by 1
- 2021Behavior of Keyword Spotting Networks Under Noisy Conditionscited by 0
- 2022A 14uJ/Decision Keyword Spotting Accelerator with In-SRAM-Computing and On Chip Learning for Customizationcited by 0
- 2023Semi-Supervised Federated Learning for Keyword Spottingcited by 0
- 2023Efficient Spoken Language Recognition via Multilabel Classificationcited by 0
- 2024A Configurable and Efficient Memory Hierarchy for Neural Network Hardware Acceleratorcited by 0
- 2024Enhancing User Experience in On-Device Machine Learning with Gated Compression Layerscited by 0
- 2024RepCNN: Micro-sized, Mighty Models for Wakeword Detectioncited by 0
- 2024Energy-Efficient Seizure Detection Suitable for low-power Applicationscited by 0
- 2024Advancing Airport Tower Command Recognition: Integrating Squeeze-and-Excitation and Broadcasted Residual Learningcited by 0
- 2024Multitaper mel-spectrograms for keyword spottingcited by 0
- 2024Automatic Generation of Fast and Accurate Performance Models for Deep Neural Network Acceleratorscited by 0
- 2025Vocal Tract Length Warped Features for Spoken Keyword Spottingcited by 0
- 2025PatchDSU: Uncertainty Modeling for Out of Distribution Generalization in Keyword Spottingcited by 0
- 2025Keyword Mamba: Spoken Keyword Spotting with State Space Modelscited by 0
- 2025Transformer Redesign for Late Fusion of Audio-Text Features on Ultra-Low-Power Edge Hardwarecited by 0
- 2026EdgeSpot: Efficient and High-Performance Few-Shot Model for Keyword Spottingcited by 0
- 2026PCOV-KWS: Multi-task Learning for Personalized Customizable Open Vocabulary Keyword Spottingcited by 0
- 2026Pmeta-TLA: Backdoor Attacks for Speech Classification Models via Meta-Learning with Timbre Leakage Attackcited by 0
- 2026DRL-CLBA: A Clean Label Backdoor Attack for Speech Classification via DDPG Reinforcement Learningcited by 0
- 2026Cumsum-Composable Phase Transport for Low-Cost Streaming Keyword Spottingcited by 0
- 2026Cumsum-Composable Phase Transport for Low-Cost Streaming Keyword Spotting22 Jul 2026
- 2026DRL-CLBA: A Clean Label Backdoor Attack for Speech Classification via DDPG Reinforcement Learning2 Jul 2026
- 2026Pmeta-TLA: Backdoor Attacks for Speech Classification Models via Meta-Learning with Timbre Leakage Attack2 Jul 2026
- 2026PCOV-KWS: Multi-task Learning for Personalized Customizable Open Vocabulary Keyword Spotting5 Mar 2026
- 2026EdgeSpot: Efficient and High-Performance Few-Shot Model for Keyword Spotting22 Jan 2026
- 2025Transformer Redesign for Late Fusion of Audio-Text Features on Ultra-Low-Power Edge Hardware20 Oct 2025
- 2025Keyword Mamba: Spoken Keyword Spotting with State Space Models10 Aug 2025
- 2025PatchDSU: Uncertainty Modeling for Out of Distribution Generalization in Keyword Spotting5 Aug 2025
- 2025AnalyticKWS: Towards Exemplar-Free Analytic Class Incremental Learning for Small-footprint Keyword Spotting17 May 2025
- 2025Vocal Tract Length Warped Features for Spoken Keyword Spotting7 Jan 2025
- 2024Automatic Generation of Fast and Accurate Performance Models for Deep Neural Network Accelerators13 Sep 2024
- 2024Multitaper mel-spectrograms for keyword spotting5 Jul 2024
37 more
- 2024Advancing Airport Tower Command Recognition: Integrating Squeeze-and-Excitation and Broadcasted Residual Learning26 Jun 2024
- 2024Energy-Efficient Seizure Detection Suitable for low-power Applications19 Jun 2024
- 2024ED-sKWS: Early-Decision Spiking Neural Networks for Rapid,and Energy-Efficient Keyword Spotting14 Jun 2024
- 2024RepCNN: Micro-sized, Mighty Models for Wakeword Detection4 Jun 2024
- 2024Enhancing User Experience in On-Device Machine Learning with Gated Compression Layers2 May 2024
- 2024A Configurable and Efficient Memory Hierarchy for Neural Network Hardware Accelerator24 Apr 2024
- 2023VIC-KD: Variance-Invariance-Covariance Knowledge Distillation to Make Keyword Spotting More Robust Against Adversarial Attacks22 Sep 2023
- 2023Federated Few-shot Learning for Cough Classification with Edge Devices3 Sep 2023
- 2023Efficient Spoken Language Recognition via Multilabel Classification2 Jun 2023
- 2023Semi-Supervised Federated Learning for Keyword Spotting9 May 2023
- 2023To Wake-up or Not to Wake-up: Reducing Keyword False Alarm by Successive Refinement6 Apr 2023
- 2023Exploring Representation Learning for Small-Footprint Keyword Spotting20 Mar 2023
- 2023Lightweight Neural Architecture Search for Temporal Convolutional Networks at the Edge24 Jan 2023
- 2022LiCo-Net: Linearized Convolution Network for Hardware-efficient Keyword Spotting9 Nov 2022
- 2022TC-SKNet with GridMask for Low-complexity Classification of Acoustic scene5 Oct 2022
- 2022Position Prediction as an Effective Pretraining Strategy15 Jul 2022
- 2022Learning Audio-Text Agreement for Open-vocabulary Keyword Spotting30 Jun 2022
- 2022Personalized Keyword Spotting through Multi-task Learning28 Jun 2022
- 2022A 14uJ/Decision Keyword Spotting Accelerator with In-SRAM-Computing and On Chip Learning for Customization10 May 2022
- 2022Improving Voice Trigger Detection with Metric Learning5 Apr 2022
- 2022Learning Decoupling Features Through Orthogonality Regularization31 Mar 2022
- 2022Rainbow Keywords: Efficient Incremental Learning for Online Spoken Keyword Spotting30 Mar 2022
- 2022Filler Word Detection and Classification: A Dataset and Benchmark28 Mar 2022
- 2022Neural Architecture Search for Energy Efficient Always-on Audio Models9 Feb 2022
- 2022Progressive Continual Learning for Spoken Keyword Spotting29 Jan 2022
- 2021Deep Spoken Keyword Spotting: An Overview20 Nov 2021
- 2021Temporal Knowledge Distillation for On-device Audio Classification27 Oct 2021
- 2021Streaming on-device detection of device directed speech from voice and touch-based invocation9 Oct 2021
- 2021FakeWake: Understanding and Mitigating Fake Wake-up Words of Voice Assistants21 Sep 2021
- 2021Behavior of Keyword Spotting Networks Under Noisy Conditions15 Sep 2021
- 2021Text Anchor Based Metric Learning for Small-footprint Keyword Spotting12 Aug 2021
- 2021Broadcasted Residual Learning for Efficient Keyword Spotting8 Jun 2021
- 2021Widening Access to Applied Machine Learning with TinyML7 Jun 2021
- 2021Encoder-Decoder Neural Architecture Optimization for Keyword Spotting4 Jun 2021
- 2021A Streaming End-to-End Framework For Spoken Language Understanding20 May 2021
- 2021Keyword Transformer: A Self-Attention Model for Keyword Spotting1 Apr 2021
- 2021SubSpectral Normalization for Neural Audio Data Processing25 Mar 2021
- 2021Keyword Transformer: A Self-Attention Model for Keyword Spottingcited by 60
- 2021Broadcasted Residual Learning for Efficient Keyword Spottingcited by 48
- 2021Deep Spoken Keyword Spotting: An Overviewcited by 43
- 2022Position Prediction as an Effective Pretraining Strategycited by 24
- 2022Learning Audio-Text Agreement for Open-vocabulary Keyword Spottingcited by 20
- 2021Widening Access to Applied Machine Learning with TinyMLcited by 17
- 2023Lightweight Neural Architecture Search for Temporal Convolutional Networks at the Edgecited by 15
- 2022TC-SKNet with GridMask for Low-complexity Classification of Acoustic scenecited by 12
- 2021SubSpectral Normalization for Neural Audio Data Processingcited by 11
- 2021A Streaming End-to-End Framework For Spoken Language Understandingcited by 8
- 2022Progressive Continual Learning for Spoken Keyword Spottingcited by 8
- 2022Rainbow Keywords: Efficient Incremental Learning for Online Spoken Keyword Spottingcited by 8
12 most cited of 44 All 44 (JSON)
- 2026Cumsum-Composable Phase Transport for Low-Cost Streaming Keyword Spotting22 Jul 2026
- 2026PCOV-KWS: Multi-task Learning for Personalized Customizable Open Vocabulary Keyword Spotting5 Mar 2026
- 2026EdgeSpot: Efficient and High-Performance Few-Shot Model for Keyword Spotting22 Jan 2026
- 2025Transformer Redesign for Late Fusion of Audio-Text Features on Ultra-Low-Power Edge Hardware20 Oct 2025
- 2025PatchDSU: Uncertainty Modeling for Out of Distribution Generalization in Keyword Spotting5 Aug 2025
- 2025AnalyticKWS: Towards Exemplar-Free Analytic Class Incremental Learning for Small-footprint Keyword Spotting17 May 2025
- 2025Vocal Tract Length Warped Features for Spoken Keyword Spotting7 Jan 2025
- 2024Automatic Generation of Fast and Accurate Performance Models for Deep Neural Network Accelerators13 Sep 2024
- 2024Multitaper mel-spectrograms for keyword spotting5 Jul 2024
- 2024Advancing Airport Tower Command Recognition: Integrating Squeeze-and-Excitation and Broadcasted Residual Learning26 Jun 2024
- 2024Energy-Efficient Seizure Detection Suitable for low-power Applications19 Jun 2024
- 2024ED-sKWS: Early-Decision Spiking Neural Networks for Rapid,and Energy-Efficient Keyword Spotting14 Jun 2024
12 newest of 44 All 44 (JSON)