# MSA-CNN: A Lightweight Multi-Scale CNN with Attention for Sleep Stage Classification

arXiv: https://arxiv.org/abs/2501.02949 · HTML: https://mlsanity.com/paper/2501.02949/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 55 references that are in the map.
- First posted on arXiv 6 Jan 2025.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces](https://mlsanity.com/paper/1611.08024/) (2016)
- [DeepSleepNet: a Model for Automatic Sleep Stage Scoring based on Raw Single-Channel EEG](https://mlsanity.com/paper/1703.04046/) (2017)
- [Model Complexity of Deep Learning: A Survey](https://mlsanity.com/paper/2103.05127/) (2021)
- [A deep learning architecture for temporal sleep stage classification using multivariate and multimodal time series](https://mlsanity.com/paper/1707.03321/) (2017)
- [Automatic Sleep Staging of EEG Signals: Recent Development, Challenges, and Future Directions](https://mlsanity.com/paper/2111.08446/) (2021)
- [Multi-View Spatial-Temporal Graph Convolutional Networks with Domain Generalization for Sleep Stage Classification](https://mlsanity.com/paper/2109.01824/) (2021)
- [Depthwise Separable Convolutions for Neural Machine Translation](https://mlsanity.com/paper/1706.03059/) (2017)
- [DeepSleepNet-Lite: A Simplified Automatic Sleep Stage Scoring Model with Uncertainty Estimates](https://mlsanity.com/paper/2108.10600/) (2021)
- [Fully-Connected Spatial-Temporal Graph for Multivariate Time-Series Data](https://mlsanity.com/paper/2309.05305/) (2023)
- [Automatic Sleep Stage Scoring with Single-Channel EEG Using Convolutional Neural Networks](https://mlsanity.com/paper/1610.01683/) (2016)
- [Multi-Scale Neural network for EEG Representation Learning in BCI](https://mlsanity.com/paper/2003.02657/) (2020)
- [ML Interpretability: Simple Isn't Easy](https://mlsanity.com/paper/2211.13617/) (2022)
- [A CNN-Transformer Deep Learning Model for Real-time Sleep Stage Classification in an Energy-Constrained Wireless Device](https://mlsanity.com/paper/2211.13005/) (2022)

## Built on by (most cited first, 3 of 3)

- [ULW-SleepNet: An Ultra-Lightweight Network for Multimodal Sleep Stage Scoring](https://mlsanity.com/paper/2602.23852/) (2026)
- [PRISM: Lightweight Multivariate Time-Series Classification through Symmetric Multi-Resolution Convolutional Layers](https://mlsanity.com/paper/2508.04503/) (2025)
- [Efficient Temporal Modeling for Mobile Sleep Staging via Lightweight Random Attention](https://mlsanity.com/paper/2606.13694/) (2026)

## Abstract (arXiv)

Recent advancements in machine learning-based signal analysis, coupled with open data initiatives, have fuelled efforts in automatic sleep stage classification. Despite the proliferation of classification models, few have prioritised reducing model complexity, which is a crucial factor for practical applications. In this work, we introduce Multi-Scale and Attention Convolutional Neural Network (MSA-CNN), a lightweight architecture featuring as few as ~10,000 parameters. MSA-CNN leverages a novel multi-scale module employing complementary pooling to eliminate redundant filter parameters and dense convolutions. Model complexity is further reduced by separating temporal and spatial feature extraction and using cost-effective global spatial convolutions. This separation of tasks not only reduces model complexity but also mirrors the approach used by human experts in sleep stage scoring. We evaluated both small and large configurations of MSA-CNN against nine state-of-the-art baseline models across three public datasets, treating univariate and multivariate models separately. Our evaluation, based on repeated cross-validation and re-evaluation of all baseline models, demonstrated that the large MSA-CNN outperformed all baseline models on all three datasets in terms of accuracy and Cohen's kappa, despite its significantly reduced parameter count. Lastly, we explored various model variants and conducted an in-depth analysis of the key modules and techniques, providing deeper insights into the underlying mechanisms. The code for our models, baselines, and evaluation procedures is available at https://github.com/sgoerttler/MSA-CNN.