mlsanityData to 14 Oct 2026

Voice-Light: A Full-Duplex Cascaded Voice Agent with Causal Turn-Taking and Speculative Generation

arXiv 2609.20995 ↗First posted 17 Sep 2026cs.SD cs.CL eess.AScited by 0

Not on the map. Why?

Abstract (arXiv)

Natural spoken interaction requires more than streaming ASR, language generation, and speech synthesis: a system must react to overlap without canceling on every acknowledgment, prepare a response before a turn is certain, and ensure canceled audio cannot enter conversation history. We present Voice-Light, a full-duplex cascaded voice agent that combines immediate acoustic onset, a causal adapter sharing a streaming ASR encoder, reversible playback control, and private speculative response generation. Structured tool calls execute concurrently with audible bridge speech, while browser acknowledgments make rendered audio authoritative for durable history. Locked evaluation on 1,673 real-conversation silence candidates found that an earlier learned completion checkpoint preserved a 2.70% false-cutoff rate but reached only 12.53% end-of-turn recall, compared with 95.60% for a Silero timing policy. The deployed system therefore retains a hybrid controller rather than claiming a learned-policy replacement. Across three unscripted operator-run microphone sessions, 36 measured response turns had a 758 ms median from final VAD endpoint to first server audio; 21 turns were below 800 ms. These sessions are an instrumented case study, not a controlled user evaluation. We release the synthetic data, model artifacts, evaluation code and summaries, source code, and deployment configuration supporting the result.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 13 of its 24 references that are in the map. First posted on arXiv 17 Sep 2026.

Builds on

13 of 24 references.

References · 13

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  3. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  4. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 568
  5. 2025CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-trainingcited by 150
  6. 2023Fast Conformer with Linearly Scalable Attention for Efficient Speech Recognitioncited by 124
  7. 2026Qwen3-TTS Technical Reportcited by 118
  8. 2022Voice Activity Projection: Self-supervised Learning of Turn-taking Eventscited by 60
  9. 2020TurnGPT: a Transformer-based Language Model for Predicting Turn-taking in Spoken Dialogcited by 56
  10. 2025Full-Duplex-Bench v1.5: Evaluating Overlap Handling for Full-Duplex Speech Modelscited by 54
  11. 2023Stateful Conformer with Cache-based Inference for Streaming Automatic Speech Recognitioncited by 25
  12. 2024Turn-taking and Backchannel Prediction with Acoustic and Large Language Model Fusioncited by 20
1 more
  1. 2025Speculative End-Turn Detector for Efficient Speech Chatbot Assistantcited by 6

Built on by

Not cited yet.