# Speech Slytherin: Examining the Performance and Efficiency of Mamba for Speech Separation, Recognition, and Synthesis

arXiv: https://arxiv.org/abs/2407.09732 · HTML: https://mlsanity.com/paper/2407.09732/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Placed in Speech and audio SSMs, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 29 of its 36 references that are in the map.
- First posted on arXiv 13 Jul 2024.

## Placed in

- State-space sequence models (S4, Mamba) / [Speech and audio SSMs](https://mlsanity.com/ssm/speech-and-audio-ssms/)

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers](https://mlsanity.com/paper/2301.02111/) (2023)
- [High Fidelity Neural Audio Compression](https://mlsanity.com/paper/2210.13438/) (2022)
- [LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech](https://mlsanity.com/paper/1904.02882/) (2019)
- [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018)
- [U-Mamba: Enhancing Long-range Dependency for Biomedical Image Segmentation](https://mlsanity.com/paper/2401.04722/) (2024)
- [Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation](https://mlsanity.com/paper/1809.07454/) (2018)
- [Jamba: A Hybrid Transformer-Mamba Language Model](https://mlsanity.com/paper/2403.19887/) (2024)
- [Deep clustering: Discriminative embeddings for segmentation and separation](https://mlsanity.com/paper/1508.04306/) (2015)
- [An Empirical Study of Mamba-based Language Models](https://mlsanity.com/paper/2406.07887/) (2024)
- [Attention is All You Need in Speech Separation](https://mlsanity.com/paper/2010.13154/) (2020)
- [Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separation](https://mlsanity.com/paper/1910.06379/) (2019)
- [Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understanding](https://mlsanity.com/paper/2403.09626/) (2024)
- [Dual-Path Transformer Network: Direct Context-Aware Modeling for End-to-End Monaural Speech Separation](https://mlsanity.com/paper/2007.13975/) (2020)
- [An Investigation of Incorporating Mamba for Speech Enhancement](https://mlsanity.com/paper/2405.06573/) (2024)
- [Mamba in Speech: Towards an Alternative to Self-Attention](https://mlsanity.com/paper/2405.12609/) (2024)
- [Dual-path Mamba: Short and Long-term Bidirectional Selective Structured State Space Models for Speech Separation](https://mlsanity.com/paper/2403.18257/) (2024)
- [SSAMBA: Self-Supervised Audio Representation Learning with Mamba State Space Model](https://mlsanity.com/paper/2405.11831/) (2024)
- [Audio Mamba: Bidirectional State Space Model for Audio Representation Learning](https://mlsanity.com/paper/2406.03344/) (2024)
- [Multichannel Long-Term Streaming Neural Speech Enhancement for Static and Moving Speakers](https://mlsanity.com/paper/2403.07675/) (2024)
- [MossFormer: Pushing the Performance Limit of Monaural Speech Separation using Gated Single-Head Transformer with Convolution-Augmented Joint Self-Attentions](https://mlsanity.com/paper/2302.11824/) (2023)
- [RawBMamba: End-to-End Bidirectional State Space Model for Audio Deepfake Detection](https://mlsanity.com/paper/2406.06086/) (2024)
- [Exploring the Capability of Mamba in Speech Applications](https://mlsanity.com/paper/2406.16808/) (2024)
- [Audio Mamba: Selective State Spaces for Self-Supervised Audio Representations](https://mlsanity.com/paper/2406.02178/) (2024)
- [TRAMBA: A Hybrid Transformer and Mamba Architecture for Practical Audio and Bone Conduction Speech Super Resolution and Enhancement on Mobile and Wearable Platforms](https://mlsanity.com/paper/2405.01242/) (2024)
- [Audio Mamba: Pretrained Audio State Space Model For Audio Tagging](https://mlsanity.com/paper/2405.13636/) (2024)

## Built on by (most cited first, 23 of 23)

- [Rethinking Mamba in Speech Processing by Self-Supervised Models](https://mlsanity.com/paper/2409.07273/) (2024)
- [Mamba for Streaming ASR Combined with Unimodal Aggregation](https://mlsanity.com/paper/2410.00070/) (2024)
- [DuplexMamba: Enhancing Real-time Speech Conversations with Duplex and Streaming Capabilities](https://mlsanity.com/paper/2502.11123/) (2025)
- [Vector Quantized Diffusion Model Based Speech Bandwidth Extension](https://mlsanity.com/paper/2409.05784/) (2024)
- [AAD-LLM: Neural Attention-Driven Auditory Scene Understanding](https://mlsanity.com/paper/2502.16794/) (2025)
- [CleanUMamba: A Compact Mamba Network for Speech Denoising using Channel Pruning](https://mlsanity.com/paper/2410.11062/) (2024)
- [Deep Active Speech Cancellation with Mamba-Masking Network](https://mlsanity.com/paper/2502.01185/) (2025)
- [An Exploration of Mamba for Speech Self-Supervised Models](https://mlsanity.com/paper/2506.12606/) (2025)
- [Selective State Space Model for Monaural Speech Enhancement](https://mlsanity.com/paper/2411.06217/) (2024)
- [MLMA: Towards Multilingual ASR With Mamba-based Architectures](https://mlsanity.com/paper/2510.18684/) (2025)
- [MamTra: A Hybrid Mamba-Transformer Backbone for Speech Synthesis](https://mlsanity.com/paper/2603.12342/) (2026)
- [Active Speech Enhancement: Active Speech Denoising Decliping and Deveraberation](https://mlsanity.com/paper/2505.16911/) (2025)
- [Recognizing Dementia from Neuropsychological Tests with State Space Models](https://mlsanity.com/paper/2507.10311/) (2025)
- [Keyword Mamba: Spoken Keyword Spotting with State Space Models](https://mlsanity.com/paper/2508.07363/) (2025)
- [An overview of neural architectures for self-supervised audio representation learning from masked spectrograms](https://mlsanity.com/paper/2509.18691/) (2025)
- [ELEGANCE: Efficient LLM Guidance for Audio-Visual Target Speech Extraction](https://mlsanity.com/paper/2511.06288/) (2025)
- [State Space Models for Bioacoustics: A Comparative Evaluation with Transformers](https://mlsanity.com/paper/2512.03563/) (2025)
- [MambaVoiceCloning: Efficient and Expressive Text-to-Speech via State-Space Modeling and Diffusion Control](https://mlsanity.com/paper/2604.00292/) (2026)
- [WAND: Windowed Attention and Knowledge Distillation for Efficient Autoregressive Text-to-Speech Models](https://mlsanity.com/paper/2604.08558/) (2026)
- [TokenSE: a Mamba-based discrete token speech enhancement framework for cochlear implants](https://mlsanity.com/paper/2604.12246/) (2026)
- [From Monolingual to Multilingual: Evaluating Mamba for ASR in South African Languages](https://mlsanity.com/paper/2607.01502/) (2026)
- [Phoneme- vs. Character-Level Targets and Selective State-Space Models for Intracortical Brain-to-Text](https://mlsanity.com/paper/2607.26751/) (2026)
- [DAVSS: Distilled Audio-Visual State Space Models](https://mlsanity.com/paper/2608.19523/) (2026)

## Abstract (arXiv)

It is too early to conclude that Mamba is a better alternative to transformers for speech before comparing Mamba with transformers in terms of both performance and efficiency in multiple speech-related tasks. To reach this conclusion, we propose and evaluate three models for three tasks: Mamba-TasNet for speech separation, ConMamba for speech recognition, and VALL-M for speech synthesis. We compare them with transformers of similar sizes in performance, memory, and speed. Our Mamba or Mamba-transformer hybrid models show comparable or higher performance than their transformer counterparts: Sepformer, Conformer, and VALL-E. They are more efficient than transformers in memory and speed for speech longer than a threshold duration, inversely related to the resolution of a speech token. Mamba for separation is the most efficient, and Mamba for recognition is the least. Further, we show that Mamba is not more efficient than transformer for speech shorter than the threshold duration and performs worse in models that require joint modeling of text and speech, such as cross or masked attention of two inputs. Therefore, we argue that the superiority of Mamba or transformer depends on particular problems and models. Code available at https://github.com/xi-j/Mamba-TasNet and https://github.com/xi-j/Mamba-ASR.