mlsanityData to 7 Oct 2026

SwanVoice: Expressive Long-Form Zero-Shot Speech Synthesis for Both Monologue and Dialogue

arXiv 2605.30993 ↗First posted 29 May 2026eess.AScited by 1

Not on the map. Why?

Abstract (arXiv)

Zero-shot text-to-speech (TTS) has improved substantially for single-speaker synthesis, yet expressive long-form multi-speaker dialogue remains difficult. A common workaround is to synthesize each turn with a monologue TTS model and stitch the outputs together. This adds inference cost and often breaks acoustic consistency, conversational coherence, and affective continuity across turns. Recent dialogue TTS systems have begun to address this setting, but they still struggle to keep expressive coherence, controllable speaker switching, and monologue quality at the same time. We present SwanData-Speech and SwanVoice. SwanData-Speech builds monologue and dialogue corpora from in-the-wild audio, using Swan Forced Aligner for pause-aware word-level alignment and RobustMegaTTS3 for pronunciation-hard cases. Built on these data, SwanVoice is a zero-shot TTS model for 1--4 speakers, combining a 25 Hz VAE, raw-text conditioning with pause-aware symbols and pinyin substitution, and a flow-matching DiT with speaker-turn conditioning. Training starts from monologue speech, moves through mixed and real dialogue data, and then uses DiffusionNFT post-training with phone-level and speaker-similarity rewards. On SwanBench-Speech, SwanVoice obtains higher richness and hierarchy scores than all evaluated open-source baselines in both monologue and dialogue settings, while content accuracy remains the main limitation. Audio demos are available at https://swanaigc.github.io//#swanvoice.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). First posted on arXiv 29 May 2026.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026SwanTale: Unified Multi-Speaker Speech and Audio Generation for Instruct and Zero-Shot Tasks
  2. 2026CuteTTS: Efficient and High-Quality Speech Synthesis via Autoregressive Modeling of Continuous Latents
  3. 2025Flamed-TTS: Flow Matching Attention-Free Models for Efficient Generating and Dynamic Pacing Zero-shot Text-to-Speech
  4. 2026Alignment-Free Text-Audiobox for Voice Dubbing and Full-Duplex Dialogue Synthesis
  5. 2024FlashSpeech: Efficient Zero-Shot Speech Synthesis
  6. 2025DialoSpeech: Dual-Speaker Dialogue Generation with LLM and Flow Matching
  7. 2026From Script to Drama: An Agentic Framework for Controllable Multi-Speaker Dialogue TTS
  8. 2026Monotonicity-Guided Semantic Alignment for Zero-shot Multispeaker Image-to-Speech Synthesis ↗