mlsanityData to 7 Oct 2026

xLSTM-SENet: xLSTM for Single-Channel Speech Enhancement

arXiv 2501.06146 ↗First posted 10 Jan 2025cs.SD cs.AI eess.AScited by 4

On the map: State-space sequence models (S4, Mamba) → Speech and audio SSMs

Abstract (arXiv)

While attention-based architectures, such as Conformers, excel in speech enhancement, they face challenges such as scalability with respect to input sequence length. In contrast, the recently proposed Extended Long Short-Term Memory (xLSTM) architecture offers linear scalability. However, xLSTM-based models remain unexplored for speech enhancement. This paper introduces xLSTM-SENet, the first xLSTM-based single-channel speech enhancement system. A comparative analysis reveals that xLSTM-and notably, even LSTM-can match or outperform state-of-the-art Mamba- and Conformer-based systems across various model sizes in speech enhancement on the VoiceBank+Demand dataset. Through ablation studies, we identify key architectural design choices such as exponential gating and bidirectionality contributing to its effectiveness. Our best xLSTM-based model, xLSTM-SENet2, outperforms state-of-the-art Mamba- and Conformer-based systems of similar complexity on the Voicebank+DEMAND dataset.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Speech and audio SSMs, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 26 of its 36 references that are in the map. First posted on arXiv 10 Jan 2025.

Builds on

26 of 36 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Audio Mamba: Selective State Spaces for Self-Supervised Audio Representations4 Jun
  2. 2024An Investigation of Incorporating Mamba for Speech Enhancement10 May
  3. 2023Spiking Structured State Space Model for Monaural Speech Enhancement7 Sep

Other references · 23

  1. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  2. 2021AST: Audio Spectrogram Transformercited by 634
  3. 2022Speech Enhancement and Dereverberation with Diffusion-based Generative Modelscited by 195
  4. 2019Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separationcited by 182
  5. 2022Conditional Diffusion Probabilistic Model for Speech Enhancementcited by 132
  6. 2021MetricGAN+: An Improved Version of MetricGAN for Speech Enhancementcited by 114
  7. 2019MetricGAN: Generative Adversarial Networks based Black-box Metric Scores Optimization for Speech Enhancementcited by 104
  8. 2023MP-SENet: A Speech Enhancement Model with Parallel Denoising of Magnitude and Phase Spectracited by 51
  9. 2024Vision-LSTM: xLSTM as Generic Vision Backbonecited by 51
  10. 2018Differentiable Consistency Constraints for Improved Deep Speech Enhancementcited by 35
  11. 2021DPT-FSNet: Dual-path Transformer Based Full-band and Sub-band Fusion Network for Speech Enhancementcited by 33
  12. 2022CMGAN: Conformer-Based Metric-GAN for Monaural Speech Enhancementcited by 22
11 more
  1. 2017Conditional Generative Adversarial Networks for Speech Enhancement and Noise-Robust Speaker Verificationcited by 19
  2. 2019On Loss Functions for Supervised Monaural Time-Domain Speech Enhancementcited by 18
  3. 2019VoiceID Loss: Speech Enhancement for Speaker Verificationcited by 10
  4. 2022Efficient Transformer-based Speech Enhancement Using Long Frames and STFT Magnitudescited by 10
  5. 2022Neural Speech Phase Prediction based on Parallel Estimation Architecture and Anti-Wrapping Lossescited by 10
  6. 2023Multi-channel Speech Separation Using Spatially Selective Deep Non-linear Filterscited by 10
  7. 2023Investigating the Design Space of Diffusion Models for Speech Enhancementcited by 10
  8. 2022SCP-GAN: Self-Correcting Discriminator Optimization for Training Consistency Preserving Metric GAN on Speech Enhancement Taskscited by 8
  9. 2022On the Role of Spatial, Spectral, and Temporal Processing for DNN-based Non-linear Multi-channel Speech Enhancementcited by 6
  10. 2022TridentSE: Guiding Speech Enhancement with 32 Global Tokenscited by 4
  11. 2024AxLSTMs: learning self-supervised audio representations with xLSTMscited by 3

Built on by

Cited by year

Citing papers per year: 2025: 4.

4 papers.

  1. 2025A Survey of Deep Learning for Complex Speech Spectrogramscited by 3
  2. 2025MambAttention: Mamba with Multi-Head Attention for Generalizable Single-Channel Speech Enhancementcited by 2
  3. 2025An overview of neural architectures for self-supervised audio representation learning from masked spectrogramscited by 0
  4. 2025Exploring Resolution-Wise Shared Attention in Hybrid Mamba-U-Nets for Improved Cross-Corpus Speech Enhancementcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Long-Context Modeling Networks for Monaural Speech Enhancement: A Comparative Study
  2. 2024xLSTM: Extended Long Short-Term Memory
  3. 2024Modulating State Space Model with SlowFast Framework for Compute-Efficient Ultra Low-Latency Speech Enhancement
  4. 2024Mamba-SEUNet: Mamba UNet for Monaural Speech Enhancement
  5. 2024MC-SEMamba: A Simple Multi-channel Extension of SEMamba
  6. 2026RT-SEMamba: Real-Time Speech Enhancement Mamba via Progressive Knowledge Distillation
  7. 2024Ultra-Low Latency Speech Enhancement - A Comprehensive Study
  8. 2024Selective State Space Model for Monaural Speech Enhancement