mlsanityData to 14 Oct 2026

Speech-Mamba: Long-Context Speech Recognition with Selective State Spaces Models

arXiv 2409.18654 ↗First posted 27 Sep 2024eess.AS cs.SDcited by 9

On the map: State-space sequence models (S4, Mamba) → Speech and audio SSMs

Abstract (arXiv)

Current automatic speech recognition systems struggle with modeling long speech sequences due to high quadratic complexity of Transformer-based models. Selective state space models such as Mamba has performed well on long-sequence modeling in natural language processing and computer vision tasks. However, research endeavors in speech technology tasks has been under-explored. We propose Speech-Mamba, which incorporates selective state space modeling in Transformer neural architectures. Long sequence representations with selective state space models in Speech-Mamba is complemented with lower-level representations from Transformer-based modeling. Speech-mamba achieves better capacity to model long-range dependencies, as it scales near-linearly with sequence length.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in Speech and audio SSMs, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 30 of its 45 references that are in the map. First posted on arXiv 27 Sep 2024.

Builds on

30 of 45 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Exploring the Capability of Mamba in Speech Applications24 Jun
  2. 2024Mamba in Speech: Towards an Alternative to Self-Attention21 May

Other references · 28

  1. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  3. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  4. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  5. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  6. 2019Root Mean Square Layer Normalizationcited by 1,480
  7. 2024Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Modelcited by 1,056
  8. 2021Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layerscited by 608
  9. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 558
  10. 2015Listen, Attend and Spellcited by 543
  11. 2024U-Mamba: Enhancing Long-range Dependency for Biomedical Image Segmentationcited by 419
  12. 2021SpeechBrain: A General-Purpose Speech Toolkitcited by 406
16 more
  1. 2024Jamba: A Hybrid Transformer-Mamba Language Modelcited by 401
  2. 2022Diagonal State Spaces are as Effective as Structured State Spacescited by 350
  3. 2024VM-UNet: Vision Mamba UNet for Medical Image Segmentationcited by 287
  4. 2024Swin-UMamba: Mamba-based UNet with ImageNet-based pretrainingcited by 179
  5. 2015End-to-End Attention-based Large Vocabulary Speech Recognitioncited by 151
  6. 2021Recent Advances in End-to-End Automatic Speech Recognitioncited by 140
  7. 2017The Microsoft 2017 Conversational Speech Recognition Systemcited by 52
  8. 2017Advances in Joint CTC-Attention based End-to-End Speech Recognition with a Deep CNN Encoder and RNN-LMcited by 43
  9. 2020End-to-End Multi-speaker Speech Recognition with Transformercited by 42
  10. 2024U-shaped Vision Mamba for Single Image Dehazingcited by 40
  11. 2016Very Deep Convolutional Networks for End-to-End Speech Recognitioncited by 34
  12. 2024Weak-Mamba-UNet: Visual Mamba Makes CNN and ViT Work Better for Scribble-based Medical Image Segmentationcited by 29
  13. 2020Simplified Self-Attention for Transformer-based End-to-End Speech Recognitioncited by 20
  14. 2022PoLyScriber: Integrated Fine-tuning of Extractor and Lyrics Transcriber for Polyphonic Musiccited by 7
  15. 2022Self-Transriber: Few-shot Lyrics Transcription with Self-trainingcited by 3
  16. 2022Music-robust Automatic Lyrics Transcription of Polyphonic Musiccited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 3.

9 papers.

  1. 2024Long-Form Speech Generation with Spoken Language Modelscited by 19
  2. 2025Fake-Mamba: Real-Time Speech Deepfake Detection Using Bidirectional Mamba as Self-Attention's Alternativecited by 6
  3. 2025An Exploration of Mamba for Speech Self-Supervised Modelscited by 2
  4. 2025MLMA: Towards Multilingual ASR With Mamba-based Architecturescited by 1
  5. 2026MamTra: A Hybrid Mamba-Transformer Backbone for Speech Synthesiscited by 1
  6. 2025Keyword Mamba: Spoken Keyword Spotting with State Space Modelscited by 0
  7. 2025An overview of neural architectures for self-supervised audio representation learning from masked spectrogramscited by 0
  8. 2026MORE: Multi-Objective Adversarial Attacks on Speech Recognitioncited by 0
  9. 2026Spiking and Event-driven Neuromorphic Mamba Models for Efficient Speech Recognitioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Dual-path Mamba: Short and Long-term Bidirectional Selective Structured State Space Models for Speech Separation
  2. 2024Mamba-based Decoder-Only Approach with Bidirectional Speech Modeling for Speech Recognition
  3. 2024Selective State Space Model for Monaural Speech Enhancement
  4. 2024An Empirical Study of Mamba-based Language Models
  5. 2024A Survey of Mamba
  6. 2026Omni-directional attention mechanism based on Mamba for speech separation
  7. 2024Mamba-ND: Selective State Space Modeling for Multi-Dimensional Data
  8. 2024Speech Slytherin: Examining the Performance and Efficiency of Mamba for Speech Separation, Recognition, and Synthesis