mlsanityData to 14 Oct 2026

Real-time and Continuous Turn-taking Prediction Using Voice Activity Projection

arXiv 2401.04868 ↗First posted 10 Jan 2024cs.CL cs.HC cs.SDcited by 21

Not on the map. Why?

Abstract (arXiv)

A demonstration of a real-time and continuous turn-taking prediction system is presented. The system is based on a voice activity projection (VAP) model, which directly maps dialogue stereo audio to future voice activities. The VAP model includes contrastive predictive coding (CPC) and self-attention transformers, followed by a cross-attention transformer. We examine the effect of the input context audio length and demonstrate that the proposed system can operate in real-time with CPU settings, with minimal performance degradation.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Builds on 3 of its 20 references that are in the map. First posted on arXiv 10 Jan 2024.

Builds on

3 of 20 references.

References · 3

  1. 2020Unsupervised pretraining transfers well across languagescited by 101
  2. 2022Voice Activity Projection: Self-supervised Learning of Turn-taking Eventscited by 60
  3. 2020TurnGPT: a Transformer-based Language Model for Predicting Turn-taking in Spoken Dialogcited by 56

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 5; 2026: 11.

21 papers.

Sort:

By area:

  1. 2024Multilingual Turn-taking Prediction Using Voice Activity Projectioncited by 26
  2. 2024Yeah, Un, Oh: Continuous and Real-time Backchannel Prediction with Fine-tuning of Voice Activity Projectioncited by 8
  3. 2026TurnBench: A Multi-Domain Benchmark for Turn-Taking Dynamics in Spoken Dialoguecited by 7
  4. 2025A Noise-Robust Turn-Taking System for Real-World Dialogue Robots: A Field Experimentcited by 6
  5. 2024Target conversation extraction: Source separation using turn-taking dynamicscited by 4
  6. 2024Human-Like Embodied AI Interviewer: Employing Android ERICA in Real International Conferencecited by 3
  7. 2025Prior Lessons of Incremental Dialogue and Robot Action Management for the Age of Language Modelscited by 2
  8. 2025Voice Activity Projection Model with Multimodal Encoderscited by 2
  9. 2024Investigating the Effects of Large-Scale Pseudo-Stereo Data and Different Speech Foundation Model on Dialogue Generative Spoken Language Modelcited by 1
  10. 2025Proactive Hearing Assistants that Isolate Egocentric Conversationscited by 1
  11. 2026JAL-Turn: Joint Acoustic-Linguistic Modeling for Real-Time and Robust Turn-Taking Detection in Full-Duplex Spoken Dialogue Systemscited by 1
  12. 2026TurnNat: Automatic Evaluation of Turn-Taking Naturalness in Dyadic Spoken Dialoguecited by 1
9 more
  1. 2025Everything Counts: The Managed Omnirelevance of Speech in Human-Voice Agent Interactioncited by 0
  2. 2026Toward Signing Activity Projection in Sign Language Interactioncited by 0
  3. 2026Fast When, Careful Who: Dual-Process Multiparty Turn-Taking with Diffusion Augmentationcited by 0
  4. 2026Multimodal Voice Activity Projection for Turn-Taking in Social Robots with Voice-Activity-Related Pretrained Encoderscited by 0
  5. 2026Low-Latency Turn-Taking via Context-Aware Preface Generation in a Real-World Dialogue Robotcited by 0
  6. 2026Real-TurnTurk: A Multimodal Turkish Corpus for Turn-Taking Predictioncited by 0
  7. 2026Exploring Multimodal Turn-Taking Cues in Face-to-Face Conversation using Voice Activity Projectioncited by 0
  8. 2026Neither Silence nor Overlap Is Failure: Intent-Conditioned Evaluation of Turn-Taking in Full-Duplex Spoken Dialogue Modelscited by 0
  9. 2026Evaluating Real-Time Voice Agents: From Component Quality to Grounded Outcomescited by 0