# Voice activity detection in the wild: A data-driven approach using teacher-student training

arXiv: https://arxiv.org/abs/2105.04065 · HTML: https://mlsanity.com/paper/2105.04065/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Speech and audio distillation, one of 12 directions in Knowledge distillation.
- Builds on 19 of its 55 references that are in the map.
- First posted on arXiv 10 May 2021.

## Placed in

- Knowledge distillation / [Speech and audio distillation](https://mlsanity.com/distill/speech-and-audio-distillation/)

## Builds on (most cited first, 19 of 19)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition](https://mlsanity.com/paper/1912.10211/) (2019)
- [MUSAN: A Music, Speech, and Noise Corpus](https://mlsanity.com/paper/1510.08484/) (2015)
- [A Comparison of Five Multiple Instance Learning Pooling Functions for Sound Event Detection with Weak Labeling](https://mlsanity.com/paper/1810.09050/) (2018)
- [rVAD: An Unsupervised Segment-Based Robust Voice Activity Detection Method](https://mlsanity.com/paper/1906.03588/) (2019)
- [A Framework for the Robust Evaluation of Sound Event Detection](https://mlsanity.com/paper/1910.08440/) (2019)
- [Personal VAD: Speaker-Conditioned Voice Activity Detection](https://mlsanity.com/paper/1908.04284/) (2019)
- [Large-scale weakly supervised audio classification using gated convolutional neural network](https://mlsanity.com/paper/1710.00343/) (2017)
- [Audio Set classification with attention model: A probabilistic perspective](https://mlsanity.com/paper/1711.00927/) (2017)
- [Large-Scale Weakly Labeled Semi-Supervised Sound Event Detection in Domestic Environments](https://mlsanity.com/paper/1807.10501/) (2018)
- [End-to-end Domain-Adversarial Voice Activity Detection](https://mlsanity.com/paper/1910.10655/) (2019)
- [Towards duration robust weakly supervised sound event detection](https://mlsanity.com/paper/2101.07687/) (2021)
- [Sound Event Detection and Time-Frequency Segmentation from Weakly Labelled Data](https://mlsanity.com/paper/1804.04715/) (2018)
- [Duration robust weakly supervised sound event detection](https://mlsanity.com/paper/1904.03841/) (2019)
- [Denoising Deep Neural Networks Based Voice Activity Detection](https://mlsanity.com/paper/1303.0663/) (2013)
- [Dual Attention in Time and Frequency Domain for Voice Activity Detection](https://mlsanity.com/paper/2003.12266/) (2020)

## Built on by (most cited first, 5 of 5)

- [Accurate Emotion Strength Assessment for Seen and Unseen Speech Based on Data-Driven Deep Learning](https://mlsanity.com/paper/2206.07229/) (2022)
- [Self-supervised Pretraining for Robust Personalized Voice Activity Detection in Adverse Conditions](https://mlsanity.com/paper/2312.16613/) (2023)
- [Efficient speech detection in environmental audio using acoustic recognition and knowledge distillation](https://mlsanity.com/paper/2312.09269/) (2023)
- [Noise-Robust Target-Speaker Voice Activity Detection Through Self-Supervised Pretraining](https://mlsanity.com/paper/2501.03184/) (2025)
- [Performance and Complexity Trade-off Optimization of Speech Models During Training](https://mlsanity.com/paper/2601.13704/) (2026)

## Abstract (arXiv)

Voice activity detection is an essential pre-processing component for speech-related tasks such as automatic speech recognition (ASR). Traditional supervised VAD systems obtain frame-level labels from an ASR pipeline by using, e.g., a Hidden Markov model. These ASR models are commonly trained on clean and fully transcribed data, limiting VAD systems to be trained on clean or synthetically noised datasets. Therefore, a major challenge for supervised VAD systems is their generalization towards noisy, real-world data. This work proposes a data-driven teacher-student approach for VAD, which utilizes vast and unconstrained audio data for training. Unlike previous approaches, only weak labels during teacher training are required, enabling the utilization of any real-world, potentially noisy dataset. Our approach firstly trains a teacher model on a source dataset (Audioset) using clip-level supervision. After training, the teacher provides frame-level guidance to a student model on an unlabeled, target dataset. A multitude of student models trained on mid- to large-sized datasets are investigated (Audioset, Voxceleb, NIST SRE). Our approach is then respectively evaluated on clean, artificially noised, and real-world data. We observe significant performance gains in artificially noised and real-world scenarios. Lastly, we compare our approach against other unsupervised and supervised VAD methods, demonstrating our method's superiority.