# SWIM: Short-Window CNN Integrated with Mamba for EEG-Based Auditory Spatial Attention Decoding

arXiv: https://arxiv.org/abs/2409.19884 · HTML: https://mlsanity.com/paper/2409.19884/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Speech and audio SSMs, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 16 of its 47 references that are in the map.
- First posted on arXiv 30 Sep 2024.

## Placed in

- State-space sequence models (S4, Mamba) / [Speech and audio SSMs](https://mlsanity.com/ssm/speech-and-audio-ssms/)

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://mlsanity.com/paper/2405.21060/) (2024)
- [It's Raw! Audio Generation with State-Space Models](https://mlsanity.com/paper/2202.09729/) (2022)
- [Data augmentation for learning predictive models on EEG: a systematic comparison](https://mlsanity.com/paper/2206.14483/) (2022)
- [EEG-based Auditory Attention Decoding: Towards Neuro-Steered Hearing Devices](https://mlsanity.com/paper/2008.04569/) (2020)
- [Learning Robust Features using Deep Learning for Automatic Seizure Detection](https://mlsanity.com/paper/1608.00220/) (2016)
- [Learning from Heterogeneous EEG Signals with Differentiable Channel Reordering](https://mlsanity.com/paper/2010.13694/) (2020)
- [A DenseNet-based method for decoding auditory spatial attention with EEG](https://mlsanity.com/paper/2309.07690/) (2023)
- [Multimodal deep learning approach for joint EEG-EMG data compression and classification](https://mlsanity.com/paper/1703.08970/) (2017)
- [NeuroNet: A Novel Hybrid Self-Supervised Learning Framework for Sleep Stage Classification Using Single-Channel EEG](https://mlsanity.com/paper/2404.17585/) (2024)
- [MSSC-BiMamba: Multimodal Sleep Stage Classification and Early Diagnosis of Sleep Disorders with Bidirectional Mamba](https://mlsanity.com/paper/2405.20142/) (2024)
- [Benchmarking Neural Decoding Backbones towards Enhanced On-edge iBCI Applications](https://mlsanity.com/paper/2406.06626/) (2024)

## Built on by (most cited first, 5 of 5)

- [Comprehensive Review of EEG-to-Output Research: Decoding Neural Signals into Images, Videos, and Audio](https://mlsanity.com/paper/2412.19999/) (2024)
- [FAConformer: Frequency-Aware Convolutional Transformer for Auditory Attention Decoding](https://mlsanity.com/paper/2606.14120/) (2026)
- [NEUROTOKEN: Joint Source and Directional AAD with Envelope Decoding via Conditional Flow Matching](https://mlsanity.com/paper/2610.00397/) (2026)
- [RAMamba-Net: A Reliability-Aware and Mamba-Based Multimodal Fusion Network for Auditory Attention Detection](https://mlsanity.com/paper/2609.11372/) (2026)
- [Merit: Making every modality earn its contribution in multimodal attention decoding](https://mlsanity.com/paper/2610.17507/) (2026)

## Abstract (arXiv)

In complex auditory environments, the human auditory system possesses the remarkable ability to focus on a specific speaker while disregarding others. In this study, a new model named SWIM, a short-window convolution neural network (CNN) integrated with Mamba, is proposed for identifying the locus of auditory attention (left or right) from electroencephalography (EEG) signals without relying on speech envelopes. SWIM consists of two parts. The first is a short-window CNN (SWCNN), which acts as a short-term EEG feature extractor and achieves a final accuracy of 84.9% in the leave-one-speaker-out setup on the widely used KUL dataset. This improvement is due to the use of an improved CNN structure, data augmentation, multitask training, and model combination. The second part, Mamba, is a sequence model first applied to auditory spatial attention decoding to leverage the long-term dependency from previous SWCNN time steps. By joint training SWCNN and Mamba, the proposed SWIM structure uses both short-term and long-term information and achieves an accuracy of 86.2%, which reduces the classification errors by a relative 31.0% compared to the previous state-of-the-art result. The source code is available at https://github.com/windowso/SWIM-ASAD.