# Adaptive Knowledge Distillation for Device-Directed Speech Detection

arXiv: https://arxiv.org/abs/2508.02801 · HTML: https://mlsanity.com/paper/2508.02801/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Speech and audio distillation, one of 12 directions in Knowledge distillation.
- Builds on 21 of its 27 references that are in the map.
- First posted on arXiv 4 Aug 2025.

## Placed in

- Knowledge distillation / [Speech and audio distillation](https://mlsanity.com/distill/speech-and-audio-distillation/)

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Compact Language Models via Pruning and Knowledge Distillation](https://mlsanity.com/paper/2407.14679/) (2024)
- [DistilHuBERT: Speech Representation Learning by Layer-wise Distillation of Hidden-unit BERT](https://mlsanity.com/paper/2110.01900/) (2021)
- [Knowledge Distillation with the Reused Teacher Classifier](https://mlsanity.com/paper/2203.14001/) (2022)
- [Distilling Knowledge from Graph Convolutional Networks](https://mlsanity.com/paper/2003.10477/) (2020)
- [WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkit](https://mlsanity.com/paper/2102.01547/) (2021)
- [Branchformer: Parallel MLP-Attention Architectures to Capture Local and Global Context for Speech Recognition and Understanding](https://mlsanity.com/paper/2207.02971/) (2022)
- [Distil-Whisper: Robust Knowledge Distillation via Large-Scale Pseudo Labelling](https://mlsanity.com/paper/2311.00430/) (2023)
- [DPHuBERT: Joint Distillation and Pruning of Self-Supervised Speech Models](https://mlsanity.com/paper/2305.17651/) (2023)
- [FitHuBERT: Going Thinner and Deeper for Knowledge Distillation of Speech Self-Supervised Learning](https://mlsanity.com/paper/2207.00555/) (2022)
- [Device-directed Utterance Detection](https://mlsanity.com/paper/1808.02504/) (2018)
- [SELMA: A Speech-Enabled Language Model for Virtual Assistant Interactions](https://mlsanity.com/paper/2501.19377/) (2025)
- [Recycle-and-Distill: Universal Compression Strategy for Transformer-based Speech SSL Models with Attention Map Reusing and Masking Distillation](https://mlsanity.com/paper/2305.11685/) (2023)
- [Device-Directed Speech Detection: Regularization via Distillation for Weakly-Supervised Models](https://mlsanity.com/paper/2203.15975/) (2022)
- [Multichannel Voice Trigger Detection Based on Transform-average-concatenate](https://mlsanity.com/paper/2309.16036/) (2023)
- [A Global-local Attention Framework for Weakly Labelled Audio Tagging](https://mlsanity.com/paper/2102.01931/) (2021)

## Built on by (most cited first, 3 of 3)

- [Trust the uncertain teacher: distilling dark knowledge via calibrated uncertainty](https://mlsanity.com/paper/2602.12687/) (2026)
- [Selective Attention System (SAS): Device-Addressed Speech Detection for Real-Time On-Device Voice AI](https://mlsanity.com/paper/2604.08412/) (2026)
- [From Acoustic Sensing to Physical Actions: A Comprehensive Survey of Security Risks in Edge-Deployed Audio LLM-Based Agents](https://mlsanity.com/paper/2610.15390/) (2026)

## Abstract (arXiv)

Device-directed speech detection (DDSD) is a binary classification task that separates the user's queries to a voice assistant (VA) from background speech or side conversations. This is important for achieving naturalistic user experience. To this end, we propose knowledge distillation (KD) to enhance DDSD accuracy while ensuring efficient deployment. Specifically, we introduce a novel adaptive KD method that transfers knowledge from general representations of an ASR large pre-trained acoustic encoder (teacher). We apply task-specific adapters, on top of the (frozen) teacher encoder, trained jointly with the student model on DDSD. We demonstrate that the proposed adaptive KD outperforms the student model without distillation in the keyword and keyword-free (follow-up) invocations, with an improvement of +26% and +19% in terms of Equal Error Rate, respectively. We also show that this approach generalizes across the transformer and conformer-based model architectures.