# Liberating LLM Capabilities in Full-Duplex Speech Models

arXiv: https://arxiv.org/abs/2606.07547 · HTML: https://mlsanity.com/paper/2606.07547/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 29 references that are in the map.
- First posted on arXiv 4 May 2026.

## Builds on (most cited first, 22 of 22)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [MiniCPM-V: A GPT-4V Level MLLM on Your Phone](https://mlsanity.com/paper/2408.01800/) (2024)
- [Moshi: a speech-text foundation model for real-time dialogue](https://mlsanity.com/paper/2410.00037/) (2024)
- [Qwen3-Omni Technical Report](https://mlsanity.com/paper/2509.17765/) (2025)
- [CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models](https://mlsanity.com/paper/2412.10117/) (2024)
- [Kimi-Audio Technical Report](https://mlsanity.com/paper/2504.18425/) (2025)
- [VoiceBench: Benchmarking LLM-Based Voice Assistants](https://mlsanity.com/paper/2410.17196/) (2024)
- [Step-Audio 2 Technical Report](https://mlsanity.com/paper/2507.16632/) (2025)
- [Language Model Can Listen While Speaking](https://mlsanity.com/paper/2408.02622/) (2024)
- [Step-Audio-R1 Technical Report](https://mlsanity.com/paper/2511.15848/) (2025)
- [URO-Bench: Towards Comprehensive Evaluation for End-to-End Spoken Dialogue Models](https://mlsanity.com/paper/2502.17810/) (2025)
- [STITCH: Simultaneous Thinking and Talking with Chunked Reasoning for Spoken Language Models](https://mlsanity.com/paper/2507.15375/) (2025)
- [Mini-Omni-Reasoner: Token-Level Thinking-in-Speaking in Large Speech Models](https://mlsanity.com/paper/2508.15827/) (2025)
- [SHANKS: Simultaneous Hearing and Thinking for Spoken Language Models](https://mlsanity.com/paper/2510.06917/) (2025)
- [Can Speech LLMs Think while Listening?](https://mlsanity.com/paper/2510.07497/) (2025)
- [FlexDuo: A Pluggable System for Enabling Full-Duplex Capabilities in Speech Dialogue Systems](https://mlsanity.com/paper/2502.13472/) (2025)
- [Chronological Thinking in Full-Duplex Spoken Dialogue Language Models](https://mlsanity.com/paper/2510.05150/) (2025)
- [Mind-Paced Speaking: A Dual-Brain Approach to Real-Time Reasoning in Spoken Language Models](https://mlsanity.com/paper/2510.09592/) (2025)
- [DrVoice: Parallel Speech-Text Voice Conversation Model via Dual-Resolution Speech Representations](https://mlsanity.com/paper/2506.09349/) (2025)
- [Think, Verbalize, then Speak: Bridging Complex Thoughts and Comprehensible Speech](https://mlsanity.com/paper/2509.16028/) (2025)

## Built on by (most cited first, 2 of 2)

- [LLMs are General Asynchronous Agents](https://mlsanity.com/paper/2609.35427/) (2026)
- [Full-duplex Spoken Language Modeling with Discrete Diffusion](https://mlsanity.com/paper/2610.17780/) (2026)

## Abstract (arXiv)

Speech-based large language models are typically constrained to spoken replies, which limits their user-facing outputs to what can be verbalized and suppresses text-native capabilities such as code generation, structured analysis, and multi-step reasoning in realtime interaction, for tasks that require persistent, structured, and inspectable intermediate outputs. Existing work improves spoken reasoning or full-duplex turn-taking, but still treats text as a hidden intermediate state or a subordinate modality rather than a first-class output channel. We propose Listen-Write-Speak (LWS), a text-first tri-channel paradigm in which a single autoregressive LLM continuously listens to user audio, writes visible free-form text as its primary output, and speaks a realtime oral response in parallel under a shared causal attention context. This behavior is implemented entirely through a Token Schema, requiring no architectural modifications, and learned via a two-stage data pipeline that synthesizes per-second cognitive annotations consistent with the revealed input timeline. Empirically, LWS demonstrates strong full-duplex interaction on Full-Duplex-Bench, reaches 4.72 on VoiceBench AlpacaEval, achieves 92.6% writing-speaking consistency, and consistently outperforms its internal ablations on URO-Bench. These results suggest that visible writing can serve as a first-class output channel for speech interaction without sacrificing realtime responsiveness. The code and dataset are available on the project page: https://royalzhang.com/project/lws-page/.