D2: High-Capability Full-Duplex Spoken Language Models from Half-Duplex Backbones
Not on the map. Why?
Abstract (arXiv)
Full-duplex spoken language models (FD-SLMs) support simultaneous listening and speaking, but often lag behind half-duplex spoken language models (HD-SLMs) in general capabilities. We propose Dialogue-to-Duplex (D2), a framework that converts pretrained HD-SLMs into high-capability FD-SLMs at fine temporal granularity. D2 adapts the original audio encoder to streaming through decoder-free representation distillation and learns full-duplex interaction through supervised fine-tuning. We systematically study the effects of chunk size on full-duplex interaction, general capabilities, and inference computation. We apply D2 to two distinct HD-SLM architectures, Qwen3-Omni and LLaMA-Omni 2. At a 12.5Hz update rate, Qwen3-Omni-D2 achieves 99.2% turn-taking success with 521-ms latency, 99.5% interruption success, and a state-of-the-art VoiceBench score of 70.1.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 31 of its 38 references that are in the map. First posted on arXiv 12 Oct 2026.
Builds on
31 of 38 references.
References · 31
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
- 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
- 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
- 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
- 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
- 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
- 2024Moshi: a speech-text foundation model for real-time dialoguecited by 568
- 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 529
- 2025Qwen3-Omni Technical Reportcited by 497
19 more
- 2023AudioPaLM: A Large Language Model That Can Speak and Listencited by 343
- 2024GLM-4-Voice: Towards Intelligent and Human-Like End-to-End Spoken Chatbotcited by 224
- 2024Mini-Omni: Language Models Can Hear, Talk While Thinking in Streamingcited by 210
- 2024VoiceBench: Benchmarking LLM-Based Voice Assistantscited by 200
- 2026Qwen3.5-Omni Technical Reportcited by 174
- 2024VITA: Towards Open-Source Interactive Omni Multimodal LLMcited by 173
- 2022SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualizationcited by 168
- 2021DistilHuBERT: Speech Representation Learning by Layer-wise Distillation of Hidden-unit BERTcited by 144
- 2024Freeze-Omni: A Smart and Low Latency Speech-to-speech Dialogue Model with Frozen LLMcited by 141
- 2021WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkitcited by 125
- 2020Emformer: Efficient Memory Transformer Based Acoustic Model For Low Latency Streaming Speech Recognitioncited by 113
- 2025Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilitiescited by 100
- 2026MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interactioncited by 78
- 2025LLaMA-Omni2: LLM-based Real-time Spoken Chatbot with Autoregressive Streaming Speech Synthesiscited by 76
- 2024Language Model Can Listen While Speakingcited by 63
- 2026PersonaPlex: Voice and Role Control for Full Duplex Conversational Speech Modelscited by 63
- 2024OmniFlatten: An End-to-end GPT Model for Seamless Voice Conversationcited by 60
- 2021SD-QA: Spoken Dialectal Question Answering for the Real Worldcited by 38
- 2026BayLing-Duplex: Native Full-Duplex Speech Dialogue with a Single Autoregressive LLMcited by 6
Built on by
Not cited yet.