DiffuPlex: Accelerating Full-Duplex Spoken Dialog Models via Rolling Masked Diffusion
On the map: Diffusion models → Audio, speech and music also fits: Discrete and text diffusion
Abstract (arXiv)
Recent full-duplex spoken dialog models enable simultaneous listening and speaking, but fine-grained models still advance their backbone autoregressively at every interaction frame. We introduce DiffuPlex, a rolling masked diffusion framework that reduces this sequential computation by predicting multiple future user and assistant frames in a single backbone wake. DiffuPlex consumes only a confident prefix of each predicted future while interaction continues at the original frame rate. As user speech arrives, it checks the corresponding user predictions and, when the interaction diverges, preserves already played assistant content while revising only the unplayed future. We consider two inference policies over the same predictor: DiffuPlex-LISTEN consumes multiple future frames when they predict assistant silence, whereas DiffuPlex-SPEAK can also consume predicted assistant speech. Across full-duplex interaction and spoken-language evaluations, DiffuPlex substantially reduces sequential backbone computation while largely preserving interaction behavior and general capability. DiffuPlex-LISTEN and DiffuPlex-SPEAK achieve 1.46× and 1.59× deployment-path wall-clock speedups and 1.61× and 1.80× Core LM speedups, with all measured backbone invocations completing within the 80ms interaction interval. Human evaluation shows that LISTEN preserves speech naturalness and conversational quality, while SPEAK retains conversational quality with some degradation in speech naturalness.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 22 of its 63 references that are in the map. First posted on arXiv 8 Oct 2026.
Builds on
22 of 63 references.
Closest ancestors · 2
Same direction, newest first
- 2026DELTA-TTS: Adapting Autoregressive Model into Diffusion Language Model for Text-to-Speech5 Jul
- 2026VocalNet-MDM: Accelerating Streaming Speech LLM via Self-Distilled Masked Diffusion Modeling9 Feb
Other references · 20
- 2017Decoupled Weight Decay Regularizationcited by 1,831
- 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 1,006
- 2025OpenAI GPT-5 System Cardcited by 989
- 2024Moshi: a speech-text foundation model for real-time dialoguecited by 568
- 2024GLM-4-Voice: Towards Intelligent and Human-Like End-to-End Spoken Chatbotcited by 224
- 2026Qwen3-ASR Technical Reportcited by 118
- 2026MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interactioncited by 78
- 2025Baichuan-Audio: A Unified Framework for End-to-End Speech Interactioncited by 68
- 2025Your LLM Knows the Future: Uncovering Its Multi-Token Prediction Potentialcited by 47
- 2025Seamless Interaction: Dyadic Audiovisual Motion Modeling and Large-Scale Datasetcited by 40
- 2026On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stabilitycited by 35
- 2026Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decodingcited by 24
8 more
- 2025Set Block Decoding is a Language Model Inference Acceleratorcited by 23
- 2025FlexDuo: A Pluggable System for Enabling Full-Duplex Capabilities in Speech Dialogue Systemscited by 15
- 2026Multi-Faceted Interactivity Alignment in Full-Duplex Speech Modelscited by 10
- 2026Raon-Speech Technical Reportcited by 7
- 2026ASPIRin: Action Space Projection for Interactivity-Optimized Reinforcement Learning in Full-Duplex Speech Language Modelscited by 3
- 2026Chatterbox-Flash: Prior-Calibrated Block Diffusion for Streaming Zero-Shot TTScited by 3
- 2026Decoupling Conversational Dynamics in Full-Duplex Spoken Models through Reinforcement Learningcited by 3
- 2026RelayS2S: A Dual-Path Speculative Generation for Real-Time Dialoguecited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Full-duplex Spoken Language Modeling with Discrete Diffusion
- 2026Overcoming State Inertia in Full-Duplex Spoken Language Models via Activation Steering
- 2026Cocktail-Talker: Multi-Speaker Dialog Modeling in Noisy Social Environments with Turn Action GRPO
- 2026BayLing-Duplex: Native Full-Duplex Speech Dialogue with a Single Autoregressive LLM
- 2025Audio MultiChallenge: A Multi-Turn Evaluation of Spoken Dialogue Systems on Natural Human Interaction
- 2026τ-Voice: Benchmarking Full-Duplex Voice Agents on Real-World Domains
- 2024Beyond Turn-Based Interfaces: Synchronous LLMs as Full-Duplex Dialogue Agents
- 2026ResDiffFRG: Residual Diffusion for Multiple Appropriate Facial Reaction Generation