# Exploring attention mechanism for acoustic-based classification of speech utterances into system-directed and non-system-directed

arXiv: https://arxiv.org/abs/1902.00570 · HTML: https://mlsanity.com/paper/1902.00570/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Feb 2019.

## Built on by (most cited first, 10 of 10)

- [Attention, please! A survey of Neural Attention Models in Deep Learning](https://mlsanity.com/paper/2103.16775/) (2021)
- [Streaming Transformer for Hardware Efficient Voice Trigger Detection and False Trigger Mitigation](https://mlsanity.com/paper/2105.06598/) (2021)
- [SELMA: A Speech-Enabled Language Model for Virtual Assistant Interactions](https://mlsanity.com/paper/2501.19377/) (2025)
- [Device-Directed Speech Detection: Regularization via Distillation for Weakly-Supervised Models](https://mlsanity.com/paper/2203.15975/) (2022)
- [Dual-Encoder Architecture with Encoder Selection for Joint Close-Talk and Far-Talk Speech Recognition](https://mlsanity.com/paper/2109.08744/) (2021)
- [Audio-to-Intent Using Acoustic-Textual Subword Representations from End-to-End ASR](https://mlsanity.com/paper/2210.12134/) (2022)
- [Streaming Intended Query Detection using E2E Modeling for Continued Conversation](https://mlsanity.com/paper/2208.13322/) (2022)
- [Implicit Acoustic Echo Cancellation for Keyword Spotting and Device-Directed Speech Detection](https://mlsanity.com/paper/2111.10639/) (2021)
- [Turbo your multi-modal classification with contrastive learning](https://mlsanity.com/paper/2409.09282/) (2024)
- [Device Directedness with Contextual Cues for Spoken Dialog Systems](https://mlsanity.com/paper/2211.13280/) (2022)

## Abstract (arXiv)

Voice controlled virtual assistants (VAs) are now available in smartphones, cars, and standalone devices in homes. In most cases, the user needs to first "wake-up" the VA by saying a particular word/phrase every time he or she wants the VA to do something. Eliminating the need for saying the wake-up word for every interaction could improve the user experience. This would require the VA to have the capability to detect the speech that is being directed at it and respond accordingly. In other words, the challenge is to distinguish between system-directed and non-system-directed speech utterances. In this paper, we present a number of neural network architectures for tackling this classification problem based on using only acoustic features. These architectures are based on using convolutional, recurrent and feed-forward layers. In addition, we investigate the use of an attention mechanism applied to the output of the convolutional and the recurrent layers. It is shown that incorporating the proposed attention mechanism into the models always leads to significant improvement in classification accuracy. The best model achieved equal error rates of 16.25 and 15.62 percents on two distinct realistic datasets.