# NiSNN-A: Non-iterative Spiking Neural Networks with Attention with Application to Motor Imagery EEG Classification

arXiv: https://arxiv.org/abs/2312.05643 · HTML: https://mlsanity.com/paper/2312.05643/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 84 references that are in the map.
- First posted on arXiv 9 Dec 2023.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces](https://mlsanity.com/paper/1611.08024/) (2016)
- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [The UCR Time Series Archive](https://mlsanity.com/paper/1810.07758/) (2018)
- [Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks](https://mlsanity.com/paper/1706.02609/) (2017)
- [Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccades](https://mlsanity.com/paper/1507.07629/) (2015)
- [Attention Spiking Neural Networks](https://mlsanity.com/paper/2209.13929/) (2022)
- [Parallel Spiking Neurons with High Efficiency and Ability to Learn Long-term Dependencies](https://mlsanity.com/paper/2304.12760/) (2023)
- [Membrane Potential Batch Normalization for Spiking Neural Networks](https://mlsanity.com/paper/2308.08359/) (2023)
- [Mixed Neural Network Approach for Temporal Sleep Stage Classification](https://mlsanity.com/paper/1610.06421/) (2016)
- [EEG-based Cross-Subject Driver Drowsiness Recognition with an Interpretable Convolutional Neural Network](https://mlsanity.com/paper/2107.09507/) (2021)
- [Tensor-CSPNet: A Novel Geometric Deep Learning Framework for Motor Imagery Classification](https://mlsanity.com/paper/2202.02472/) (2022)
- [MIN2Net: End-to-End Multi-Task Learning for Subject-Independent Motor Imagery EEG Classification](https://mlsanity.com/paper/2102.03814/) (2021)
- [Tuning Convolutional Spiking Neural Network with Biologically-plausible Reward Propagation](https://mlsanity.com/paper/2010.04434/) (2020)
- [Unsupervised AER Object Recognition Based on Multiscale Spatio-Temporal Features and Spiking Neurons](https://mlsanity.com/paper/1911.08261/) (2019)
- [A Spatial-channel-temporal-fused Attention for Spiking Neural Networks](https://mlsanity.com/paper/2209.10837/) (2022)
- [SSGCNet: A Sparse Spectra Graph Convolutional Network for Epileptic EEG Signal Classification](https://mlsanity.com/paper/2203.12910/) (2022)
- [Improving EEG Decoding via Clustering-based Multi-task Feature Learning](https://mlsanity.com/paper/2012.06813/) (2020)
- [End-to-end Sleep Staging with Raw Single Channel EEG using Deep Residual ConvNets](https://mlsanity.com/paper/1904.10255/) (2019)
- [A Lightweight Architecture for Real-Time Neuronal-Spike Classification](https://mlsanity.com/paper/2311.04808/) (2023)

## Abstract (arXiv)

Motor imagery, an important category in electroencephalogram (EEG) research, often intersects with scenarios demanding low energy consumption, such as portable medical devices and isolated environment operations. Traditional deep learning algorithms, despite their effectiveness, are characterized by significant computational demands accompanied by high energy usage. As an alternative, spiking neural networks (SNNs), inspired by the biological functions of the brain, emerge as a promising energy-efficient solution. However, SNNs typically exhibit lower accuracy than their counterpart convolutional neural networks (CNNs). Although attention mechanisms successfully increase network accuracy by focusing on relevant features, their integration in the SNN framework remains an open question. In this work, we combine the SNN and the attention mechanisms for the EEG classification, aiming to improve precision and reduce energy consumption. To this end, we first propose a Non-iterative Leaky Integrate-and-Fire (LIF) neuron model, overcoming the gradient issues in the traditional SNNs using the Iterative LIF neurons. Then, we introduce the sequence-based attention mechanisms to refine the feature map. We evaluated the proposed Non-iterative SNN with Attention (NiSNN-A) model on OpenBMI, a large-scale motor imagery dataset. Experiment results demonstrate that 1) our model outperforms other SNN models by achieving higher accuracy, 2) our model increases energy efficiency compared to the counterpart CNN models (i.e., by 2.27 times) while maintaining comparable accuracy.