# X-Talk: On the Underestimated Potential of Modular Speech-to-Speech Dialogue System

arXiv: https://arxiv.org/abs/2512.18706 · HTML: https://mlsanity.com/paper/2512.18706/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Dec 2025.

## Built on by (most cited first, 9 of 9)

- [Discourse-Aware Dual-Track Streaming Response for Low-Latency Spoken Dialogue Systems](https://mlsanity.com/paper/2602.23266/) (2026)
- [PolySpeech-100: A Large-Scale Benchmark for Speech Understanding Across 100+ Languages and Dialects](https://mlsanity.com/paper/2606.01016/) (2026)
- [FlashTTS: Fast Streaming TTS with MTP Acceleration and X-pred Mean Flow Distillation](https://mlsanity.com/paper/2606.09141/) (2026)
- [Thinking-while-speaking: A Controlled, Interleaved Reasoning Method for Real-Time Speech Generation](https://mlsanity.com/paper/2605.20946/) (2026)
- [Proactive for Uncertainty: Cause-Aware Error Diagnosis and Interactive Clarification for Spoken Dialogue Systems](https://mlsanity.com/paper/2605.25404/) (2026)
- [Enabling Proactive Spoken Turns via a Generalized Style-Aware Full-Duplex Framework](https://mlsanity.com/paper/2608.28630/) (2026)
- [Conversation Coach: A Voice-enabled AI System that Helps Practice Difficult Workplace Conversations](https://mlsanity.com/paper/2609.00441/) (2026)
- [Transcribe, Then Reason: Two-Pass Decomposition for Multimodal Review](https://mlsanity.com/paper/2609.18958/) (2026)
- [DuplexAgent-RSI: Recursive Harness Improvement for Full-Duplex Voice Agent Collaboration](https://mlsanity.com/paper/2610.11299/) (2026)

## Abstract (arXiv)

We present X-Talk, an open-source framework that champions a decoupled, modular design for LLM-driven speech-to-speech (S2S) systems. While the dominant trend favors end-to-end (E2E) modeling to optimize information flow, these "omni-models" often struggle to balance the competing objectives of complex speech tasks within a single network. X-Talk challenges this paradigm by demonstrating that a systematically optimized cascaded pipeline can achieve sub-second latency without sacrificing modular flexibility. Our framework seamlessly integrates specialized front-end components (e.g., VAD, speech enhancement) and diverse understanding models (e.g., ASR, emotion, and environmental sound analysis) with LLM capabilities like retrieval-augmented generation (RAG) and tool use. By revitalizing the cascaded approach, X-Talk highlights the underestimated potential of modular S2S systems and provides a robust foundation for future research and applications.