# Exploring Sequence-to-Sequence Transformer-Transducer Models for Keyword Spotting

arXiv: https://arxiv.org/abs/2211.06478 · HTML: https://mlsanity.com/paper/2211.06478/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 32 references that are in the map.
- First posted on arXiv 11 Nov 2022.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Loss](https://mlsanity.com/paper/2002.02562/) (2020)
- [Minimum Word Error Rate Training for Attention-based Sequence-to-Sequence Models](https://mlsanity.com/paper/1712.01818/) (2017)
- [Deep Spoken Keyword Spotting: An Overview](https://mlsanity.com/paper/2111.10592/) (2021)
- [Convolutional Recurrent Neural Networks for Small-Footprint Keyword Spotting](https://mlsanity.com/paper/1703.05390/) (2017)
- [Turn-to-Diarize: Online Speaker Diarization Constrained by Transformer Transducer Speaker Turn Detection](https://mlsanity.com/paper/2109.11641/) (2021)
- [End-to-End Streaming Keyword Spotting](https://mlsanity.com/paper/1812.02802/) (2018)
- [Minimum Bayes Risk Training of RNN-Transducer for End-to-End Speech Recognition](https://mlsanity.com/paper/1911.12487/) (2019)
- [Multi-task Language Modeling for Improving Speech Recognition of Rare Words](https://mlsanity.com/paper/2011.11715/) (2020)
- [End-to-End ASR-free Keyword Search from Speech](https://mlsanity.com/paper/1701.04313/) (2017)
- [Streaming Small-Footprint Keyword Spotting using Sequence-to-Sequence Models](https://mlsanity.com/paper/1710.09617/) (2017)
- [Max-Pooling Loss Training of Long Short-Term Memory Networks for Small-Footprint Keyword Spotting](https://mlsanity.com/paper/1705.02411/) (2017)
- [Learning To Detect Keyword Parts And Whole By Smoothed Max Pooling](https://mlsanity.com/paper/2001.09246/) (2020)
- [Meta-Learning for improving rare word recognition in end-to-end ASR](https://mlsanity.com/paper/2102.12624/) (2021)

## Built on by (most cited first, 2 of 2)

- [A Parameter-Efficient Learning Approach to Arabic Dialect Identification with Pre-Trained General-Purpose Speech Model](https://mlsanity.com/paper/2305.11244/) (2023)
- [Personalizing Keyword Spotting with Speaker Information](https://mlsanity.com/paper/2311.03419/) (2023)

## Abstract (arXiv)

In this paper, we present a novel approach to adapt a sequence-to-sequence Transformer-Transducer ASR system to the keyword spotting (KWS) task. We achieve this by replacing the keyword in the text transcription with a special token <kw> and training the system to detect the <kw> token in an audio stream. At inference time, we create a decision function inspired by conventional KWS approaches, to make our approach more suitable for the KWS task. Furthermore, we introduce a specific keyword spotting loss by adapting the sequence-discriminative Minimum Bayes-Risk training technique. We find that our approach significantly outperforms ASR based KWS systems. When compared with a conventional keyword spotting system, our proposal has similar performance while bringing the advantages and flexibility of sequence-to-sequence training. Additionally, when combined with the conventional KWS system, our approach can improve the performance at any operation point.