# DPSNN: Spiking Neural Network for Low-Latency Streaming Speech Enhancement

arXiv: https://arxiv.org/abs/2408.07388 · HTML: https://mlsanity.com/paper/2408.07388/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 43 references that are in the map.
- First posted on arXiv 14 Aug 2024.

## Builds on (most cited first, 24 of 24)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018)
- [Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation](https://mlsanity.com/paper/1809.07454/) (2018)
- [Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks](https://mlsanity.com/paper/2007.05785/) (2020)
- [Supervised Speech Separation Based on Deep Learning: An Overview](https://mlsanity.com/paper/1708.07524/) (2017)
- [Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separation](https://mlsanity.com/paper/1910.06379/) (2019)
- [Wave-U-Net: A Multi-Scale Neural Network for End-to-End Audio Source Separation](https://mlsanity.com/paper/1806.03185/) (2018)
- [TasNet: time-domain audio separation network for real-time, single-channel speech separation](https://mlsanity.com/paper/1711.00541/) (2017)
- [Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks](https://mlsanity.com/paper/2103.12593/) (2021)
- [Phase-aware Speech Enhancement with Deep Complex U-Net](https://mlsanity.com/paper/1903.03107/) (2019)
- [Effective and Efficient Computation with Multiple-timescale Spiking Recurrent Neural Networks](https://mlsanity.com/paper/2005.11633/) (2020)
- [Raw Waveform-based Speech Enhancement by Fully Convolutional Networks](https://mlsanity.com/paper/1703.02205/) (2017)
- [Improved Speech Enhancement with the Wave-U-Net](https://mlsanity.com/paper/1811.11307/) (2018)
- [Spiking Structured State Space Model for Monaural Speech Enhancement](https://mlsanity.com/paper/2309.03641/) (2023)
- [Complex spectrogram enhancement by convolutional neural network with multi-metrics learning](https://mlsanity.com/paper/1704.08504/) (2017)
- [STFT-Domain Neural Speech Enhancement with Very Low Algorithmic Latency](https://mlsanity.com/paper/2204.09911/) (2022)
- [The Intel Neuromorphic DNS Challenge](https://mlsanity.com/paper/2303.09503/) (2023)
- [Deep neural network Based Low-latency Speech Separation with Asymmetric analysis-Synthesis Window Pair](https://mlsanity.com/paper/2106.11794/) (2021)
- [Single Channel Speech Enhancement Using U-Net Spiking Neural Networks](https://mlsanity.com/paper/2307.14464/) (2023)
- [Dilated FCN: Listening Longer to Hear Better](https://mlsanity.com/paper/1907.11956/) (2019)
- [Trainable Adaptive Window Switching for Speech Enhancement](https://mlsanity.com/paper/1811.02438/) (2018)
- [Low-Latency Deep Clustering For Speech Separation](https://mlsanity.com/paper/1902.07033/) (2019)

## Built on by (most cited first, 5 of 5)

- [Average-Over-Time Spiking Neural Networks for Uncertainty Estimation in Regression](https://mlsanity.com/paper/2412.00278/) (2024)
- [IML-Spikeformer: Input-aware Multi-Level Spiking Transformer for Speech Processing](https://mlsanity.com/paper/2507.07396/) (2025)
- [Spiking and Event-driven Neuromorphic Mamba Models for Efficient Speech Recognition](https://mlsanity.com/paper/2606.01135/) (2026)
- [DBHN-Net: Dual-Branch Hybrid Neural Network For Low-Complexity Monaural Speech Enhancement](https://mlsanity.com/paper/2606.05911/) (2026)
- [Neuromorphic Speech Enhancement with Dual-Branch Spiking Neural Networks](https://mlsanity.com/paper/2606.23761/) (2026)

## Abstract (arXiv)

Speech enhancement (SE) improves communication in noisy environments, affecting areas such as automatic speech recognition, hearing aids, and telecommunications. With these domains typically being power-constrained and event-based while requiring low latency, neuromorphic algorithms in the form of spiking neural networks (SNNs) have great potential. Yet, current effective SNN solutions require a contextual sampling window imposing substantial latency, typically around 32ms, too long for many applications. Inspired by Dual-Path Spiking Neural Networks (DPSNNs) in classical neural networks, we develop a two-phase time-domain streaming SNN framework -- the Dual-Path Spiking Neural Network (DPSNN). In the DPSNN, the first phase uses Spiking Convolutional Neural Networks (SCNNs) to capture global contextual information, while the second phase uses Spiking Recurrent Neural Networks (SRNNs) to focus on frequency-related features. In addition, the regularizer suppresses activation to further enhance energy efficiency of our DPSNNs. Evaluating on the VCTK and Intel DNS Datasets, we demonstrate that our approach achieves the very low latency (approximately 5ms) required for applications like hearing aids, while demonstrating excellent signal-to-noise ratio (SNR), perceptual quality, and energy efficiency.