# Enabling Proactive Spoken Turns via a Generalized Style-Aware Full-Duplex Framework

arXiv: https://arxiv.org/abs/2608.28630 · HTML: https://mlsanity.com/paper/2608.28630/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 38 of its 71 references that are in the map.
- First posted on arXiv 4 Aug 2026.

## Builds on (most cited first, 30 of 38)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [MiniCPM-V: A GPT-4V Level MLLM on Your Phone](https://mlsanity.com/paper/2408.01800/) (2024)
- [Recipes for building an open-domain chatbot](https://mlsanity.com/paper/2004.13637/) (2020)
- [Moshi: a speech-text foundation model for real-time dialogue](https://mlsanity.com/paper/2410.00037/) (2024)
- [GLM-4-Voice: Towards Intelligent and Human-Like End-to-End Spoken Chatbot](https://mlsanity.com/paper/2412.02612/) (2024)
- [VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interaction](https://mlsanity.com/paper/2501.01957/) (2025)
- [MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipe](https://mlsanity.com/paper/2509.18154/) (2025)
- [TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue](https://mlsanity.com/paper/2004.06871/) (2020)
- [Freeze-Omni: A Smart and Low Latency Speech-to-speech Dialogue Model with Frozen LLM](https://mlsanity.com/paper/2411.00774/) (2024)
- [Mini-Omni2: Towards Open-source GPT-4o with Vision, Speech and Duplex Capabilities](https://mlsanity.com/paper/2410.11190/) (2024)
- [Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilities](https://mlsanity.com/paper/2503.04721/) (2025)
- [MinMo: A Multimodal Large Language Model for Seamless Voice Interaction](https://mlsanity.com/paper/2501.06282/) (2025)
- [Plug-and-Play Policy Planner for Large Language Model Powered Dialogue Agents](https://mlsanity.com/paper/2311.00262/) (2023)
- [DailyTalk: Spoken Dialogue Dataset for Conversational Text-to-Speech](https://mlsanity.com/paper/2207.01063/) (2022)
- [A Survey on Proactive Dialogue Systems: Problems, Methods, and Prospects](https://mlsanity.com/paper/2305.02750/) (2023)
- [Language Model Can Listen While Speaking](https://mlsanity.com/paper/2408.02622/) (2024)
- [PersonaPlex: Voice and Role Control for Full Duplex Conversational Speech Models](https://mlsanity.com/paper/2602.06053/) (2026)
- [OmniFlatten: An End-to-end GPT Model for Seamless Voice Conversation](https://mlsanity.com/paper/2410.17799/) (2024)
- [Full-Duplex-Bench v1.5: Evaluating Overlap Handling for Full-Duplex Speech Models](https://mlsanity.com/paper/2507.23159/) (2025)
- [Talking Turns: Benchmarking Audio Foundation Models on Turn-Taking Dynamics](https://mlsanity.com/paper/2503.01174/) (2025)
- [Full-Duplex-Bench-v2: A Multi-Turn Evaluation Framework for Duplex Dialogue Systems with an Automated Examiner](https://mlsanity.com/paper/2510.07838/) (2025)
- [OSUM-EChat: Enhancing End-to-End Empathetic Spoken Chatbot via Understanding-Driven Spoken Dialogue](https://mlsanity.com/paper/2508.09600/) (2025)
- [Turn-taking and Backchannel Prediction with Acoustic and Large Language Model Fusion](https://mlsanity.com/paper/2401.14717/) (2024)
- [NTPP: Generative Speech Language Modeling for Dual-Channel Spoken Dialogue via Next-Token-Pair Prediction](https://mlsanity.com/paper/2506.00975/) (2025)
- [Towards human-like spoken dialogue generation between AI agents from written dialogue](https://mlsanity.com/paper/2310.01088/) (2023)
- [FireRedChat: A Pluggable, Full-Duplex Voice Interaction System with Cascaded and Semi-Cascaded Implementations](https://mlsanity.com/paper/2509.06502/) (2025)
- [Turn-Taking Prediction for Natural Conversational Speech](https://mlsanity.com/paper/2208.13321/) (2022)
- [FlexDuo: A Pluggable System for Enabling Full-Duplex Capabilities in Speech Dialogue Systems](https://mlsanity.com/paper/2502.13472/) (2025)

## Abstract (arXiv)

Compared with half-duplex dialogue systems where the system waits for user turn completion before it responds, natural full-duplex dialogue systems require agents to act proactively in real time, including timely interruptions and backchannels. This creates a key challenge: improving turn timing without sacrificing response quality. To address limitations in realistic proactive turn-taking, we build a generalized style-aware full-duplex framework with three key components. Firstly, we propose LPS-TC, a Lightweight Proactive Speech Turn Controller for plug-and-play integration. It features a fine-grained action space covering both reactive and proactive turn behaviors, enabling half-duplex models with full-duplex capabilities and enhancing existing full-duplex models with superior timing control. Secondly, we construct WildTurn, a large-scale, real-world English dataset containing approximately 2,981 hours of filtered multi-turn stereo conversations from face-to-face and telephone conversations, annotated with five turn-taking and five backchanneling styles. Trained on WildTurn, LPS-TC exhibits rich spoken dynamics that are not captured by existing static full-duplex benchmarks. Thirdly, we introduce a two-tier evaluation scheme that assesses both chunk-level timing precision and turn-level interaction quality under realistic streaming constraints. Our experiments, integrating LPS-TC with half-duplex models like Qwen2.5-Omni and full-duplex models like Freeze-Omni, showcase its superior performance in timing appropriateness and response quality. Our framework also demonstrates fine-grained style controllability and strong generalizability, enabling more natural and human-like spoken interactions.