# Semantic VAD: Low-Latency Voice Activity Detection for Speech Interaction

arXiv: https://arxiv.org/abs/2305.12450 · HTML: https://mlsanity.com/paper/2305.12450/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 May 2023.

## Built on by (most cited first, 6 of 6)

- [AudioJailbreak: Jailbreak Attacks against End-to-End Large Audio-Language Models](https://mlsanity.com/paper/2505.14103/) (2025)
- [From Turn-Taking to Synchronous Dialogue: A Survey of Full-Duplex Spoken Language Models](https://mlsanity.com/paper/2509.14515/) (2025)
- [LibriheavyMix: A 20,000-Hour Dataset for Single-Channel Reverberant Multi-Talker Speech Separation, ASR and Speaker Diarization](https://mlsanity.com/paper/2409.00819/) (2024)
- [Beyond Transcripts: A Renewed Perspective on Audio Chaptering](https://mlsanity.com/paper/2602.08979/) (2026)
- [From Static Inference to Dynamic Interaction: A Survey of Streaming Large Language Models](https://mlsanity.com/paper/2603.04592/) (2026)
- [A Small-footprint Acoustic Echo Cancellation Solution for Mobile Full-Duplex Speech Interactions](https://mlsanity.com/paper/2508.07561/) (2025)

## Abstract (arXiv)

For speech interaction, voice activity detection (VAD) is often used as a front-end. However, traditional VAD algorithms usually need to wait for a continuous tail silence to reach a preset maximum duration before segmentation, resulting in a large latency that affects user experience. In this paper, we propose a novel semantic VAD for low-latency segmentation. Different from existing methods, a frame-level punctuation prediction task is added to the semantic VAD, and the artificial endpoint is included in the classification category in addition to the often-used speech presence and absence. To enhance the semantic information of the model, we also incorporate an automatic speech recognition (ASR) related semantic loss. Evaluations on an internal dataset show that the proposed method can reduce the average latency by 53.3% without significant deterioration of character error rate in the back-end ASR compared to the traditional VAD approach.