mlsanityData to 14 Oct 2026

MambaVoiceCloning: Efficient and Expressive Text-to-Speech via State-Space Modeling and Diffusion Control

arXiv 2604.00292 ↗First posted 31 Mar 2026cs.SD cs.LGcited by 0

On the map: State-space sequence models (S4, Mamba) → Speech and audio SSMs

Abstract (arXiv)

MambaVoiceCloning (MVC) asks whether the conditioning path of diffusion-based TTS can be made fully SSM-only at inference, removing all attention and explicit RNN-style recurrence layers across text, rhythm, and prosody, while preserving or improving quality under controlled conditions. MVC combines a gated bidirectional Mamba text encoder, a Temporal Bi-Mamba supervised by a lightweight alignment teacher discarded after training, and an Expressive Mamba with AdaLN modulation, yielding linear-time O(T) conditioning with bounded activation memory and practical finite look-ahead streaming. Unlike prior Mamba-TTS systems that remain hybrid at inference, MVC removes attention-based duration and style modules under a fixed StyleTTS2 mel-diffusion-vocoder backbone. Trained on LJSpeech/LibriTTS and evaluated on VCTK, CSS10 (ES/DE/FR), and long-form Gutenberg passages, MVC achieves modest but statistically reliable gains over StyleTTS2, VITS, and Mamba-attention hybrids in MOS/CMOS, F0 RMSE, MCD, and WER, while reducing encoder parameters to 21M and improving throughput by 1.6x. Diffusion remains the dominant latency source, but SSM-only conditioning improves memory footprint, stability, and deployability.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Speech and audio SSMs, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 24 of its 31 references that are in the map. First posted on arXiv 31 Mar 2026.

Builds on

24 of 31 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Speech Slytherin: Examining the Performance and Efficiency of Mamba for Speech Separation, Recognition, and Synthesis13 Jul
  2. 2024Exploring the Capability of Mamba in Speech Applications24 Jun
  3. 2024Mamba in Speech: Towards an Alternative to Self-Attention21 May

Other references · 21

  1. 2020Rethinking Attention with Performerscited by 1,601
  2. 2020Linformer: Self-Attention with Linear Complexitycited by 1,474
  3. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,354
  4. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 804
  5. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
  6. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 561
  7. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 490
  8. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speechcited by 447
  9. 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Modelscited by 226
  10. 2021A Survey on Neural Speech Synthesiscited by 184
  11. 2021Diff-TTS: A Denoising Diffusion Model for Text-to-Speechcited by 157
  12. 2023StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Modelscited by 152
9 more
  1. 2022ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speechcited by 149
  2. 2023A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AIcited by 68
  3. 2019CSS10: A Collection of Single Speaker Speech Datasets for 10 Languagescited by 54
  4. 2021PnG BERT: Augmented BERT on Phonemes and Graphemes for Neural TTScited by 52
  5. 2022DiffGAN-TTS: High-Fidelity and Efficient Text-to-Speech with Denoising Diffusion GANscited by 43
  6. 2021Parallel Tacotron 2: A Non-Autoregressive Neural TTS Model with Differentiable Duration Modelingcited by 31
  7. 2022JETS: Jointly Training FastSpeech2 and HiFi-GAN for End to End Text to Speechcited by 26
  8. 2023Phoneme-Level BERT for Enhanced Prosody of Text-to-Speech with Grapheme Predictionscited by 23
  9. 2024Vision Transformer Segmentation for Visual Bird Sound Denoisingcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Speak, Edit, Repeat: High-Fidelity Voice Editing and Zero-Shot TTS with Cross-Attentive Mamba
  2. 2026LongCat-AudioDiT: High-Fidelity Diffusion Text-to-Speech in the Waveform Latent Space
  3. 2026CuteTTS: Efficient and High-Quality Speech Synthesis via Autoregressive Modeling of Continuous Latents
  4. 2026MamTra: A Hybrid Mamba-Transformer Backbone for Speech Synthesis
  5. 2025From Text to Talk: Audio-Language Model Needs Non-Autoregressive Joint Training
  6. 2026WAND: Windowed Attention and Knowledge Distillation for Efficient Autoregressive Text-to-Speech Models
  7. 2026LLaDA-TTS: Unifying Speech Synthesis and Zero-Shot Editing via Masked Diffusion Modeling
  8. 2023High-Fidelity Speech Synthesis with Minimal Supervision: All Using Diffusion Models