mlsanityData to 7 Oct 2026

Liberating LLM Capabilities in Full-Duplex Speech Models

arXiv 2606.07547 ↗First posted 4 May 2026cs.CL cs.AI cs.SDcited by 1

Not on the map. Why?

Abstract (arXiv)

Speech-based large language models are typically constrained to spoken replies, which limits their user-facing outputs to what can be verbalized and suppresses text-native capabilities such as code generation, structured analysis, and multi-step reasoning in realtime interaction, for tasks that require persistent, structured, and inspectable intermediate outputs. Existing work improves spoken reasoning or full-duplex turn-taking, but still treats text as a hidden intermediate state or a subordinate modality rather than a first-class output channel. We propose Listen-Write-Speak (LWS), a text-first tri-channel paradigm in which a single autoregressive LLM continuously listens to user audio, writes visible free-form text as its primary output, and speaks a realtime oral response in parallel under a shared causal attention context. This behavior is implemented entirely through a Token Schema, requiring no architectural modifications, and learned via a two-stage data pipeline that synthesizes per-second cognitive annotations consistent with the revealed input timeline. Empirically, LWS demonstrates strong full-duplex interaction on Full-Duplex-Bench, reaches 4.72 on VoiceBench AlpacaEval, achieves 92.6% writing-speaking consistency, and consistently outperforms its internal ablations on URO-Bench. These results suggest that visible writing can serve as a first-class output channel for speech interaction without sacrificing realtime responsiveness. The code and dataset are available on the project page: https://royalzhang.com/project/lws-page/.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 22 of its 29 references that are in the map. First posted on arXiv 4 May 2026.

Builds on

22 of 29 references.

References · 22

  1. 2025Qwen3 Technical Reportcited by 8,195
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  3. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  4. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,061
  5. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 536
  6. 2025Qwen3-Omni Technical Reportcited by 451
  7. 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Modelscited by 282
  8. 2025Kimi-Audio Technical Reportcited by 210
  9. 2024VoiceBench: Benchmarking LLM-Based Voice Assistantscited by 192
  10. 2025Step-Audio 2 Technical Reportcited by 101
  11. 2024Language Model Can Listen While Speakingcited by 62
  12. 2025Step-Audio-R1 Technical Reportcited by 35
10 more
  1. 2025URO-Bench: Towards Comprehensive Evaluation for End-to-End Spoken Dialogue Modelscited by 29
  2. 2025STITCH: Simultaneous Thinking and Talking with Chunked Reasoning for Spoken Language Modelscited by 25
  3. 2025Mini-Omni-Reasoner: Token-Level Thinking-in-Speaking in Large Speech Modelscited by 20
  4. 2025SHANKS: Simultaneous Hearing and Thinking for Spoken Language Modelscited by 16
  5. 2025FlexDuo: A Pluggable System for Enabling Full-Duplex Capabilities in Speech Dialogue Systemscited by 15
  6. 2025Can Speech LLMs Think while Listening?cited by 15
  7. 2025Chronological Thinking in Full-Duplex Spoken Dialogue Language Modelscited by 13
  8. 2025Mind-Paced Speaking: A Dual-Brain Approach to Real-Time Reasoning in Spoken Language Modelscited by 9
  9. 2025DrVoice: Parallel Speech-Text Voice Conversation Model via Dual-Resolution Speech Representationscited by 6
  10. 2025Think, Verbalize, then Speak: Bridging Complex Thoughts and Comprehensible Speechcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.