# KABURI-TTS: Phoneme-Keyed Activity-conditioned Bi-channel Utterance Rendering for Interaction

arXiv: https://arxiv.org/abs/2609.07200 · HTML: https://mlsanity.com/paper/2609.07200/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 16 references that are in the map.
- First posted on arXiv 7 Sep 2026.

## Builds on (most cited first, 10 of 10)

- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Moshi: a speech-text foundation model for real-time dialogue](https://mlsanity.com/paper/2410.00037/) (2024)
- [Generative Spoken Dialogue Language Modeling](https://mlsanity.com/paper/2203.16502/) (2022)
- [PersonaPlex: Voice and Role Control for Full Duplex Conversational Speech Models](https://mlsanity.com/paper/2602.06053/) (2026)
- [Towards human-like spoken dialogue generation between AI agents from written dialogue](https://mlsanity.com/paper/2310.01088/) (2023)
- [CoVoMix: Advancing Zero-Shot Speech Generation for Human-like Multi-talker Conversations](https://mlsanity.com/paper/2404.06690/) (2024)
- [Semantic-VAE: Semantic-Alignment Latent Representation for Better Speech Synthesis](https://mlsanity.com/paper/2509.22167/) (2025)
- [ZipVoice-Dialog: Non-Autoregressive Spoken Dialogue Generation with Flow Matching](https://mlsanity.com/paper/2507.09318/) (2025)
- [DialoSpeech: Dual-Speaker Dialogue Generation with LLM and Flow Matching](https://mlsanity.com/paper/2510.08373/) (2025)
- [MOSS-TTSD: Text to Spoken Dialogue Generation](https://mlsanity.com/paper/2603.19739/) (2026)

## Abstract (arXiv)

Realizing full-duplex spoken dialogue requires large amounts of two-channel, one-speaker-per-channel conversational speech data. Although conversational text-to-speech (TTS) engines have been developed, they are not necessarily robust to two-party simultaneous phenomena such as backchannels, interruptions, and overlaps that occur while the interlocutor is speaking. In this work, aiming at conversational speech synthesis that reproduces human-like overlap, we propose KABURI-TTS. KABURI-TTS takes a per-speaker phoneme raster as input and renders the speech of the two speakers on separate channels, conditioned on the per-frame phonemes and the voice activity derived from them. Because the phoneme raster is supplied by a separate module, the proposed method enables controllable generation of one-speaker-per-channel, two-party spoken dialogue. A user evaluation shows that, compared with strong baselines, the proposed method attains higher naturalness at both the utterance and the interaction level. Furthermore, an analysis of voice activity confirms that the proposed method produces more overlap and more frequent turn-taking.