mlsanityData to 7 Oct 2026

Exploring the Capability of Mamba in Speech Applications

arXiv 2406.16808 ↗First posted 24 Jun 2024cs.SD eess.AScited by 19

On the map: State-space sequence models (S4, Mamba) → Speech and audio SSMs

Abstract (arXiv)

This paper explores the capability of Mamba, a recently proposed architecture based on state space models (SSMs), as a competitive alternative to Transformer-based models. In the speech domain, well-designed Transformer-based models, such as the Conformer and E-Branchformer, have become the de facto standards. Extensive evaluations have demonstrated the effectiveness of these Transformer-based models across a wide range of speech tasks. In contrast, the evaluation of SSMs has been limited to a few tasks, such as automatic speech recognition (ASR) and speech synthesis. In this paper, we compared Mamba with state-of-the-art Transformer variants for various speech applications, including ASR, text-to-speech, spoken language understanding, and speech summarization. Experimental evaluations revealed that Mamba achieves comparable or better performance than Transformer-based models, and demonstrated its efficiency in long-form speech processing.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Placed in Speech and audio SSMs, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 38 of its 52 references that are in the map. First posted on arXiv 24 Jun 2024.

Builds on

38 of 52 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Augmenting conformers with structured state-space sequence models for online speech recognition15 Sep
  2. 2023A Multi-dimensional Deep Structured State Space Approach to Speech Enhancement Using Small-footprint Models1 Jun
  3. 2022Structured State Space Decoder for Speech Recognition and Synthesis31 Oct

Other references · 35

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2016Layer Normalizationcited by 5,579
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  5. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  6. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  7. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  8. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  9. 2024Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Modelcited by 1,051
  10. 2022Simplified State Space Layers for Sequence Modelingcited by 776
  11. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  12. 2022On the Parameterization and Initialization of Diagonal State Space Modelscited by 418
23 more
  1. 2021GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audiocited by 391
  2. 2017AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baselinecited by 364
  3. 2021FNet: Mixing Tokens with Fourier Transformscited by 343
  4. 2019A Comparative Study on Transformer vs RNN in Speech Applicationscited by 210
  5. 2020SLURP: A Spoken Language Understanding Resource Packagecited by 197
  6. 2023Seamless: Multilingual Expressive and Streaming Speech Translationcited by 190
  7. 2022It's Raw! Audio Generation with State-Space Modelscited by 171
  8. 2018How2: A Large-scale Dataset for Multimodal Language Understandingcited by 137
  9. 2023End-to-End Speech Recognition: A Surveycited by 127
  10. 2017Sequence-to-Sequence Models Can Directly Translate Foreign Speechcited by 121
  11. 2022Branchformer: Parallel MLP-Attention Architectures to Capture Local and Global Context for Speech Recognition and Understandingcited by 90
  12. 2022E-Branchformer: Branchformer with Enhanced merging for speech recognitioncited by 90
  13. 2023Zipformer: A faster and better encoder for automatic speech recognitioncited by 77
  14. 2021SLUE: New Benchmark Tasks for Spoken Language Understanding Evaluation on Natural Speechcited by 70
  15. 2024MambaByte: Token-free Selective State Space Modelcited by 58
  16. 2021ESPnet-SLU: Advancing Spoken Language Understanding through ESPnetcited by 53
  17. 2023Diagonal State Space Augmented Transformers for Speech Recognitioncited by 25
  18. 2021Speech Summarization using Restricted Self-Attentioncited by 23
  19. 2021ESPnet2-TTS: Extending the Edge of TTS Researchcited by 22
  20. 2022A Study on the Integration of Pre-trained SSL, ASR, LM and SLU Models for Spoken Language Understandingcited by 18
  21. 2023Multi-Head State Space Model for Speech Recognitioncited by 14
  22. 2023A Comparative Study on E-Branchformer vs Conformer in Speech Recognition, Translation, and Understanding Taskscited by 13
  23. 2023SummaryMixing: A Linear-Complexity Alternative to Self-Attention for Speech Recognition and Understandingcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 8; 2026: 4.

19 papers.

Sort:

By area:

  1. 2024Speech Slytherin: Examining the Performance and Efficiency of Mamba for Speech Separation, Recognition, and Synthesiscited by 22
  2. 2024Long-Form Speech Generation with Spoken Language Modelscited by 19
  3. 2024Speech-Mamba: Long-Context Speech Recognition with Selective State Spaces Modelscited by 9
  4. 2025Advancing Intelligent Sequence Modeling: Evolution, Trade-offs, and Applications of State-Space Architectures from S4 to Mambacited by 9
  5. 2024Mamba-based Decoder-Only Approach with Bidirectional Speech Modeling for Speech Recognitioncited by 5
  6. 2025Summarizing Speech: A Comprehensive Surveycited by 5
  7. 2025Overflow Prevention Enhances Long-Context Recurrent LLMscited by 5
  8. 2024Cross-attention Inspired Selective State Space Models for Target Sound Extractioncited by 4
  9. 2024Vector Quantized Diffusion Model Based Speech Bandwidth Extensioncited by 4
  10. 2025Quantizing Small-Scale State-Space Models for Edge AIcited by 3
  11. 2024Selective State Space Model for Monaural Speech Enhancementcited by 1
  12. 2025Developing multilingual speech synthesis system for Ojibwe, Mi'kmaq, and Maliseetcited by 1
7 more
  1. 2025MLMA: Towards Multilingual ASR With Mamba-based Architecturescited by 1
  2. 2026Omni-directional attention mechanism based on Mamba for speech separationcited by 1
  3. 2025Accurate, fast, cheap: Choose three. Replacing Multi-Head-Attention with Bidirectional Recurrent Attention for Long-Form ASRcited by 0
  4. 2025Non-Intrusive Binaural Speech Intelligibility Prediction Using Mamba for Hearing-Impaired Listenerscited by 0
  5. 2026MambaVoiceCloning: Efficient and Expressive Text-to-Speech via State-Space Modeling and Diffusion Controlcited by 0
  6. 2026From Monolingual to Multilingual: Evaluating Mamba for ASR in South African Languagescited by 0
  7. 2026Phoneme- vs. Character-Level Targets and Selective State-Space Models for Intracortical Brain-to-Textcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Mamba in Speech: Towards an Alternative to Self-Attention
  2. 2024An Empirical Study of Mamba-based Language Models
  3. 2024A Survey of Mamba
  4. 2025An Exploration of Mamba for Speech Self-Supervised Models
  5. 2024Mamba-SEUNet: Mamba UNet for Monaural Speech Enhancement
  6. 2024An Investigation of Incorporating Mamba for Speech Enhancement
  7. 2025Keyword Mamba: Spoken Keyword Spotting with State Space Models
  8. 2025Samba-ASR: State-Of-The-Art Speech Recognition Leveraging Structured State-Space Models