DuplexSLA: A Full-Duplex Spoken Language Model with Synchronized Speech, Language, and Action
Not on the map. Why?
Abstract (arXiv)
Recent advances in spoken dialogue language models have shifted from turn-based to full-duplex designs, where the model continuously listens to the user while generating responses. However, existing duplex backbones still lack a native channel for in-conversation planning and tool calling, leaving real-time agentic behaviour either tied to turn boundaries or relegated to an external cascade. We propose DuplexSLA, a native full-duplex Speech-Language-Action foundation model that decodes assistant audio together with a structured action stream on a shared 160 ms chunk timeline. DuplexSLA is built on a dual-stream three-channel formulation: a continuous user audio channel, a discrete assistant audio channel, and a rate-limited textual action channel, all decoded jointly by a single backbone, so that listening, speaking, planning, and tool calling unfold on one shared clock. Two capabilities define the model: (1) semantic-driven turn-taking control, where interruption, pause, and backchannel are handled inside the same backbone instead of by an external semantic VAD; and (2) in-conversation planning and tool calling, where planning text and structured tool calls are emitted on the action channel without halting assistant audio, so that multi-action and backchannel-triggered tool use are interleaved with ongoing speech. To evaluate these capabilities together, we further construct DuplexSLA-Bench, a duplex benchmark covering pause, interrupt, and backchannel turn-taking together with three styles of in-conversation tool calling. Our project page, interactive demos, and the DuplexSLA-Bench evaluation suite are publicly available at https://github.com/hyzhang24/DuplexSLA.
Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). First posted on arXiv 20 May 2026.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2026: 9.
9 papers.
- 2026Wan-Streamer v0.1: End-to-end Real-time Interactive Foundation Modelscited by 7
- 2026Audio Interaction Modelcited by 4
- 2026Realtime-Venus: A full-duplex interaction system with asynchronous delegationcited by 4
- 2026NemotronLabs VoiceChat: An Open Full-duplex Speech-to-Speech Model with Tool Calling Capabilitiescited by 4
- 2026A frontend-backend architecture for tool calls in full-duplex speech modelscited by 2
- 2026SALMONN-duo: Adaptive Dual-System Coordination for Full-Duplex Voice Agentscited by 1
- 2026Context Spanning: A Communication Framework for Full-Duplex Speech Models and External LLM Backendscited by 0
- 2026DuplexAct-Bench: Broadening Full-Duplex Speech Evaluation toward Proactive Interaction across Diverse Behavioral Requirementscited by 0
- 2026RASSG: Repair-Aware Semantic Survival Gating for Streaming Tool Executioncited by 0
- 2026RASSG: Repair-Aware Semantic Survival Gating for Streaming Tool Execution10 Oct 2026
- 2026DuplexAct-Bench: Broadening Full-Duplex Speech Evaluation toward Proactive Interaction across Diverse Behavioral Requirements30 Sep 2026
- 2026SALMONN-duo: Adaptive Dual-System Coordination for Full-Duplex Voice Agents28 Sep 2026
- 2026Context Spanning: A Communication Framework for Full-Duplex Speech Models and External LLM Backends27 Sep 2026
- 2026NemotronLabs VoiceChat: An Open Full-duplex Speech-to-Speech Model with Tool Calling Capabilities18 Sep 2026
- 2026A frontend-backend architecture for tool calls in full-duplex speech models16 Sep 2026
- 2026Realtime-Venus: A full-duplex interaction system with asynchronous delegation12 Sep 2026
- 2026Wan-Streamer v0.1: End-to-end Real-time Interactive Foundation Models23 Jun 2026
- 2026Audio Interaction Model3 Jun 2026