# Unified Keyword Spotting and Audio Tagging on Mobile Devices with Transformers

arXiv: https://arxiv.org/abs/2303.01812 · HTML: https://mlsanity.com/paper/2303.01812/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Mar 2023.

## Built on by (most cited first, 4 of 4)

- [Joint Audio and Speech Understanding](https://mlsanity.com/paper/2309.14405/) (2023)
- [CED: Consistent ensemble distillation for audio tagging](https://mlsanity.com/paper/2308.11957/) (2023)
- [Robust Wake-Up Word Detection by Two-stage Multi-resolution Ensembles](https://mlsanity.com/paper/2310.11379/) (2023)
- [A Literature Review of Keyword Spotting Technologies for Urdu](https://mlsanity.com/paper/2409.16317/) (2024)

## Abstract (arXiv)

Keyword spotting (KWS) is a core human-machine-interaction front-end task for most modern intelligent assistants. Recently, a unified (UniKW-AT) framework has been proposed that adds additional capabilities in the form of audio tagging (AT) to a KWS model. However, previous work did not consider the real-world deployment of a UniKW-AT model, where factors such as model size and inference speed are more important than performance alone. This work introduces three mobile-device deployable models named Unified Transformers (UiT). Our best model achieves an mAP of 34.09 on Audioset, and an accuracy of 97.76 on the public Google Speech Commands V1 dataset. Further, we benchmark our proposed approaches on four mobile platforms, revealing that the proposed UiT models can achieve a speedup of 2 - 6 times against a competitive MobileNetV2.