# Spiking and Event-driven Neuromorphic Mamba Models for Efficient Speech Recognition

arXiv: https://arxiv.org/abs/2606.01135 · HTML: https://mlsanity.com/paper/2606.01135/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Speech and audio SSMs, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 22 of its 34 references that are in the map.
- First posted on arXiv 31 May 2026.

## Placed in

- State-space sequence models (S4, Mamba) / [Speech and audio SSMs](https://mlsanity.com/ssm/speech-and-audio-ssms/)

## Builds on (most cited first, 22 of 22)

- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Efficient Processing of Deep Neural Networks: A Tutorial and Survey](https://mlsanity.com/paper/1703.09039/) (2017)
- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [Spikformer: When Spiking Neural Network Meets Transformer](https://mlsanity.com/paper/2209.15425/) (2022)
- [Training Spiking Neural Networks Using Lessons From Deep Learning](https://mlsanity.com/paper/2109.12894/) (2021)
- [Spike-driven Transformer](https://mlsanity.com/paper/2307.01694/) (2023)
- [Accurate and Structured Pruning for Efficient Automatic Speech Recognition](https://mlsanity.com/paper/2305.19549/) (2023)
- [SpikMamba: When SNN meets Mamba in Event-based Human Action Recognition](https://mlsanity.com/paper/2410.16746/) (2024)
- [Speech-Mamba: Long-Context Speech Recognition with Selective State Spaces Models](https://mlsanity.com/paper/2409.18654/) (2024)
- [Event-based Optical Flow on Neuromorphic Processor: ANN vs. SNN Comparison based on Activation Sparsification](https://mlsanity.com/paper/2407.20421/) (2024)
- [Conformer-Based Speech Recognition On Extreme Edge-Computing Devices](https://mlsanity.com/paper/2312.10359/) (2023)
- [DPSNN: Spiking Neural Network for Low-Latency Streaming Speech Enhancement](https://mlsanity.com/paper/2408.07388/) (2024)
- [PyCARL: A PyNN Interface for Hardware-Software Co-Simulation of Spiking Neural Network](https://mlsanity.com/paper/2003.09696/) (2020)
- [An Exploration of Mamba for Speech Self-Supervised Models](https://mlsanity.com/paper/2506.12606/) (2025)
- [An Event-Based Digital Compute-In-Memory Accelerator with Flexible Operand Resolution and Layer-Wise Weight/Output Stationarity](https://mlsanity.com/paper/2410.23082/) (2024)
- [Explore Activation Sparsity in Recurrent LLMs for Energy-Efficient Neuromorphic Computing](https://mlsanity.com/paper/2501.16337/) (2025)
- [AMPLE: Event-Driven Accelerator for Mixed-Precision Inference of Graph Neural Networks](https://mlsanity.com/paper/2502.21196/) (2025)
- [IML-Spikeformer: Input-aware Multi-Level Spiking Transformer for Speech Processing](https://mlsanity.com/paper/2507.07396/) (2025)
- [Context-aware Sparse Spatiotemporal Learning for Event-based Vision](https://mlsanity.com/paper/2508.19806/) (2025)

## Abstract (arXiv)

Deep learning has greatly advanced automatic speech recognition (ASR), enabling widespread deployment on edge devices such as smartphones and smart home systems. However, the computational and energy demands of deep neural networks pose significant challenges for such resource-constrained deployments, introducing latency and limiting real-time interaction. Neuromorphic computing offers a promising solution by introducing activation sparsity through spiking neural networks (SNNs) and event-driven neural networks, converting dense operations into sparse computations. However, a study that evaluates the hardware benefits of different neuromorphic strategies remains lacking for ASR. This paper explores spiking and event-driven neuromorphic neural networks to improve activation sparsity in the state-of-the-art SpeechMamba model for ASR. We introduce an event-driven SpeechMamba with FATReLU activation, achieving over 60% activation sparsity with less than 1% accuracy degradation on LibriSpeech. We also propose a spiking SpeechMamba that attains over 70% sparsity while using 30% fewer parameters than comparable SNNs. Finally, we develop a cycle-accurate event-driven simulator enabling flexible algorithm-hardware co-exploration, which helps us identify computational bottlenecks and yields over 10% additional efficiency improvements.