mlsanityData to 14 Oct 2026

ZipL-Dialog: Memory-Efficient Long-Form Spoken Dialog Synthesis via Latent Flow Matching

arXiv 2607.12496 ↗First posted 14 Jul 2026eess.AScited by 0

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

Zero-shot dialog TTS benefits from flow-matching, but minute-scale generation on dense mel-spectrograms causes severe memory bottlenecks, often forcing unnatural chunked synthesis. We propose ZipL-Dialog, which shifts conditional flow-matching into a 4x time-compressed (25 Hz) latent space. To preserve acoustic fidelity under compression, we employ a deterministic mel autoencoder with auxiliary mel-domain supervision and optimize the ZipFormer's hierarchical downsampling schedule. Experiments show that ZipL-Dialog reduces maximum peak GPU memory by 11.22x and accelerates inference by 2.23x over the baseline, substantially lowering the memory footprint of single-pass multi-minute dialog synthesis while maintaining perceptual naturalness.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 22 of its 24 references that are in the map. First posted on arXiv 14 Jul 2026.

Builds on

22 of 24 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025M3-TTS: Multi-modal DiT Alignment & Mel-latent for Zero-shot High-fidelity Speech Synthesis4 Dec
  2. 2025ZipVoice-Dialog: Non-Autoregressive Spoken Dialogue Generation with Flow Matching12 Jul
  3. 2025CoVoMix2: Advancing Zero-Shot Dialogue Generation with Fully Non-Autoregressive Flow Matching1 Jun
  4. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching9 Oct

Other references · 18

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  2. 2022Flow Matching for Generative Modelingcited by 5,702
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  4. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  5. 2017DailyDialog: A Manually Labelled Multi-turn Dialogue Datasetcited by 652
  6. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
  7. 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 628
  8. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 390
  9. 2024Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generationcited by 204
  10. 2019pyannote.audio: neural building blocks for speaker diarizationcited by 181
  11. 2023Flow Matching in Latent Spacecited by 160
  12. 2023Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesiscited by 153
6 more
  1. 2023LibriSpeech-PC: Benchmark for Evaluation of Punctuation and Capitalization Capabilities of end-to-end ASR Modelscited by 49
  2. 2025FireRedTTS-2: Towards Long Conversational Speech Generation for Podcast and Chatbotcited by 35
  3. 2025VibeVoice Technical Reportcited by 29
  4. 2024CoVoMix: Advancing Zero-Shot Speech Generation for Human-like Multi-talker Conversationscited by 15
  5. 2025HiFiTTS-2: A Large-Scale High Bandwidth Speech Datasetcited by 10
  6. 2025SoulX-Podcast: Towards Realistic Long-form Podcasts with Dialectal and Paralinguistic Diversitycited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024StreamingDialogue: Prolonged Dialogue Learning via Long Context Compression with Minimal Losses
  2. 2026CuteTTS: Efficient and High-Quality Speech Synthesis via Autoregressive Modeling of Continuous Latents
  3. 2026FastSLM: Hierarchical Temporal Abstraction for Efficient Long-Form Speech Adaptation
  4. 2026VoxZip: Semantic-Anchored Temporal KV Cache Compression for Long-Context Audio Inference
  5. 2025READ: Real-time and Efficient Asynchronous Diffusion for Audio-driven Talking Head Generation
  6. 2026Context-Driven Incremental Compression for Multi-Turn Dialogue Generation
  7. 2025CLEAR: Continuous Latent Autoregressive Modeling for High-quality and Low-latency Speech Synthesis
  8. 2026DiTAR+: Dual Optimization for Robust Autoregressive Diffusion Speech Synthesis