mlsanityData to 14 Oct 2026

Speech Slytherin: Examining the Performance and Efficiency of Mamba for Speech Separation, Recognition, and Synthesis

arXiv 2407.09732 ↗First posted 13 Jul 2024eess.AS cs.LG cs.SDcited by 23

On the map: State-space sequence models (S4, Mamba) → Speech and audio SSMs

Abstract (arXiv)

It is too early to conclude that Mamba is a better alternative to transformers for speech before comparing Mamba with transformers in terms of both performance and efficiency in multiple speech-related tasks. To reach this conclusion, we propose and evaluate three models for three tasks: Mamba-TasNet for speech separation, ConMamba for speech recognition, and VALL-M for speech synthesis. We compare them with transformers of similar sizes in performance, memory, and speed. Our Mamba or Mamba-transformer hybrid models show comparable or higher performance than their transformer counterparts: Sepformer, Conformer, and VALL-E. They are more efficient than transformers in memory and speed for speech longer than a threshold duration, inversely related to the resolution of a speech token. Mamba for separation is the most efficient, and Mamba for recognition is the least. Further, we show that Mamba is not more efficient than transformer for speech shorter than the threshold duration and performs worse in models that require joint modeling of text and speech, such as cross or masked attention of two inputs. Therefore, we argue that the superiority of Mamba or transformer depends on particular problems and models. Code available at https://github.com/xi-j/Mamba-TasNet and https://github.com/xi-j/Mamba-ASR.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Placed in Speech and audio SSMs, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 29 of its 36 references that are in the map. First posted on arXiv 13 Jul 2024.

Builds on

29 of 36 references.

Closest ancestors · 10

Same direction, newest first

  1. 2024Exploring the Capability of Mamba in Speech Applications24 Jun
  2. 2024RawBMamba: End-to-End Bidirectional State Space Model for Audio Deepfake Detection10 Jun
  3. 2024Audio Mamba: Bidirectional State Space Model for Audio Representation Learning5 Jun
  4. 2024Audio Mamba: Selective State Spaces for Self-Supervised Audio Representations4 Jun
  5. 2024Audio Mamba: Pretrained Audio State Space Model For Audio Tagging22 May
  6. 2024Mamba in Speech: Towards an Alternative to Self-Attention21 May
  7. 2024SSAMBA: Self-Supervised Audio Representation Learning with Mamba State Space Model20 May
  8. 2024An Investigation of Incorporating Mamba for Speech Enhancement10 May
  9. 2024TRAMBA: A Hybrid Transformer and Mamba Architecture for Practical Audio and Bone Conduction Speech Super Resolution and Enhancement on Mobile and Wearable Platforms2 May
  10. 2024Dual-path Mamba: Short and Long-term Bidirectional Selective Structured State Space Models for Speech Separation27 Mar

Other references · 19

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  3. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  4. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  5. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 752
  6. 2022High Fidelity Neural Audio Compressioncited by 687
  7. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
  8. 2018SDR - half-baked or well done?cited by 423
  9. 2024U-Mamba: Enhancing Long-range Dependency for Biomedical Image Segmentationcited by 419
  10. 2018Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separationcited by 406
  11. 2024Jamba: A Hybrid Transformer-Mamba Language Modelcited by 401
  12. 2015Deep clustering: Discriminative embeddings for segmentation and separationcited by 212
7 more
  1. 2024An Empirical Study of Mamba-based Language Modelscited by 186
  2. 2020Attention is All You Need in Speech Separationcited by 185
  3. 2019Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separationcited by 182
  4. 2024Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understandingcited by 108
  5. 2020Dual-Path Transformer Network: Direct Context-Aware Modeling for End-to-End Monaural Speech Separationcited by 68
  6. 2024Multichannel Long-Term Streaming Neural Speech Enhancement for Static and Moving Speakerscited by 28
  7. 2023MossFormer: Pushing the Performance Limit of Monaural Speech Separation using Gated Single-Head Transformer with Convolution-Augmented Joint Self-Attentionscited by 27

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 11; 2026: 7.

23 papers.

Sort:

By area:

  1. 2024Rethinking Mamba in Speech Processing by Self-Supervised Modelscited by 11
  2. 2024Mamba for Streaming ASR Combined with Unimodal Aggregationcited by 8
  3. 2025DuplexMamba: Enhancing Real-time Speech Conversations with Duplex and Streaming Capabilitiescited by 5
  4. 2024Vector Quantized Diffusion Model Based Speech Bandwidth Extensioncited by 4
  5. 2025AAD-LLM: Neural Attention-Driven Auditory Scene Understandingcited by 3
  6. 2024CleanUMamba: A Compact Mamba Network for Speech Denoising using Channel Pruningcited by 2
  7. 2025Deep Active Speech Cancellation with Mamba-Masking Networkcited by 2
  8. 2025An Exploration of Mamba for Speech Self-Supervised Modelscited by 2
  9. 2024Selective State Space Model for Monaural Speech Enhancementcited by 1
  10. 2025MLMA: Towards Multilingual ASR With Mamba-based Architecturescited by 1
  11. 2026MamTra: A Hybrid Mamba-Transformer Backbone for Speech Synthesiscited by 1
  12. 2025Active Speech Enhancement: Active Speech Denoising Decliping and Deveraberationcited by 0
11 more
  1. 2025Recognizing Dementia from Neuropsychological Tests with State Space Modelscited by 0
  2. 2025Keyword Mamba: Spoken Keyword Spotting with State Space Modelscited by 0
  3. 2025An overview of neural architectures for self-supervised audio representation learning from masked spectrogramscited by 0
  4. 2025ELEGANCE: Efficient LLM Guidance for Audio-Visual Target Speech Extractioncited by 0
  5. 2025State Space Models for Bioacoustics: A Comparative Evaluation with Transformerscited by 0
  6. 2026MambaVoiceCloning: Efficient and Expressive Text-to-Speech via State-Space Modeling and Diffusion Controlcited by 0
  7. 2026WAND: Windowed Attention and Knowledge Distillation for Efficient Autoregressive Text-to-Speech Modelscited by 0
  8. 2026TokenSE: a Mamba-based discrete token speech enhancement framework for cochlear implantscited by 0
  9. 2026From Monolingual to Multilingual: Evaluating Mamba for ASR in South African Languagescited by 0
  10. 2026Phoneme- vs. Character-Level Targets and Selective State-Space Models for Intracortical Brain-to-Textcited by 0
  11. 2026DAVSS: Distilled Audio-Visual State Space Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024SepMamba: State-space models for speaker separation using Mamba
  2. 2025TransMamba: A Sequence-Level Hybrid Transformer-Mamba Language Model
  3. 2024Speech-Mamba: Long-Context Speech Recognition with Selective State Spaces Models
  4. 2026Omni-directional attention mechanism based on Mamba for speech separation
  5. 2024Mamba-based Decoder-Only Approach with Bidirectional Speech Modeling for Speech Recognition
  6. 2024U-Mamba-Net: A highly efficient Mamba-based U-net style network for noisy and reverberant speech separation
  7. 2026MambAdapter: Lightweight Mamba-Based Adapters for Parameter-Efficient Transfer Learning in Speech and Audio
  8. 2024SPMamba: State-space model is all you need in speech separation