# Personal VAD: Speaker-Conditioned Voice Activity Detection

arXiv: https://arxiv.org/abs/1908.04284 · HTML: https://mlsanity.com/paper/1908.04284/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Aug 2019.

## Built on by (most cited first, 25 of 25)

- [End-to-End Active Speaker Detection](https://mlsanity.com/paper/2203.14250/) (2022)
- [DiarizationLM: Speaker Diarization Post-Processing with Large Language Models](https://mlsanity.com/paper/2401.03506/) (2024)
- [Towards Neural Diarization for Unlimited Numbers of Speakers Using Global and Local Attractors](https://mlsanity.com/paper/2107.01545/) (2021)
- [Personal VAD 2.0: Optimizing Personal Voice Activity Detection for On-Device Speech Recognition](https://mlsanity.com/paper/2204.03793/) (2022)
- [Speaker-Conditioned Hierarchical Modeling for Automated Speech Scoring](https://mlsanity.com/paper/2109.00928/) (2021)
- [Online Neural Diarization of Unlimited Numbers of Speakers Using Global and Local Attractors](https://mlsanity.com/paper/2206.02432/) (2022)
- [Configurable Privacy-Preserving Automatic Speech Recognition](https://mlsanity.com/paper/2104.00766/) (2021)
- [End-to-End Diarization for Variable Number of Speakers with Local-Global Networks and Discriminative Speaker Embeddings](https://mlsanity.com/paper/2105.02096/) (2021)
- [WASD: A Wilder Active Speaker Detection Dataset](https://mlsanity.com/paper/2303.05321/) (2023)
- [Voice activity detection in the wild: A data-driven approach using teacher-student training](https://mlsanity.com/paper/2105.04065/) (2021)
- [Multi-user VoiceFilter-Lite via Attentive Speaker Embedding](https://mlsanity.com/paper/2107.01201/) (2021)
- [End-to-End Joint Target and Non-Target Speakers ASR](https://mlsanity.com/paper/2306.02273/) (2023)
- [TS-SUPERB: A Target Speech Processing Benchmark for Speech Self-Supervised Learning Models](https://mlsanity.com/paper/2505.06660/) (2025)
- [Push-Pull: Characterizing the Adversarial Robustness for Audio-Visual Active Speaker Detection](https://mlsanity.com/paper/2210.00753/) (2022)
- [A Virtual Simulation-Pilot Agent for Training of Air Traffic Controllers](https://mlsanity.com/paper/2304.07842/) (2023)
- [In-Ear-Voice: Towards Milli-Watt Audio Enhancement With Bone-Conduction Microphones for In-Ear Sensing Platforms](https://mlsanity.com/paper/2309.02393/) (2023)
- [Self-supervised Pretraining for Robust Personalized Voice Activity Detection in Adverse Conditions](https://mlsanity.com/paper/2312.16613/) (2023)
- [SVVAD: Personal Voice Activity Detection for Speaker Verification](https://mlsanity.com/paper/2305.19581/) (2023)
- [Investigation of Speaker Representation for Target-Speaker Speech Processing](https://mlsanity.com/paper/2410.11243/) (2024)
- [Closing the Gap between Single-User and Multi-User VoiceFilter-Lite](https://mlsanity.com/paper/2202.12169/) (2022)
- [Personalizing Keyword Spotting with Speaker Information](https://mlsanity.com/paper/2311.03419/) (2023)
- [Sparsely Overlapped Speech Training in the Time Domain: Joint Learning of Target Speech Separation and Personal VAD Benefits](https://mlsanity.com/paper/2106.14371/) (2021)
- [HyWA: Architecture-Preserving Personalized Voice Activity Detection for Full-Duplex Voice Assistants](https://mlsanity.com/paper/2510.12947/) (2025)
- [LibriVAD: A Scalable Open Dataset with Deep Learning Benchmarks for Voice Activity Detection](https://mlsanity.com/paper/2512.17281/) (2025)
- [Role-guided Speaker Deletion Verification in Clinical Psychiatry Speech Recordings with Audio Language Models](https://mlsanity.com/paper/2609.38491/) (2026)

## Abstract (arXiv)

In this paper, we propose "personal VAD", a system to detect the voice activity of a target speaker at the frame level. This system is useful for gating the inputs to a streaming on-device speech recognition system, such that it only triggers for the target user, which helps reduce the computational cost and battery consumption, especially in scenarios where a keyword detector is unpreferable. We achieve this by training a VAD-alike neural network that is conditioned on the target speaker embedding or the speaker verification score. For each frame, personal VAD outputs the probabilities for three classes: non-speech, target speaker speech, and non-target speaker speech. Under our optimal setup, we are able to train a model with only 130K parameters that outperforms a baseline system where individually trained standard VAD and speaker recognition networks are combined to perform the same task.