# A CNN-Transformer Deep Learning Model for Real-time Sleep Stage Classification in an Energy-Constrained Wireless Device

arXiv: https://arxiv.org/abs/2211.13005 · HTML: https://mlsanity.com/paper/2211.13005/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 19 references that are in the map.
- First posted on arXiv 20 Nov 2022.

## Builds on (most cited first, 6 of 6)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [DeepSleepNet: a Model for Automatic Sleep Stage Scoring based on Raw Single-Channel EEG](https://mlsanity.com/paper/1703.04046/) (2017)
- [SleepEEGNet: Automated Sleep Stage Scoring with Sequence to Sequence Deep Learning Approach](https://mlsanity.com/paper/1903.02108/) (2019)
- [Edge Deep Learning for Neural Implants](https://mlsanity.com/paper/2012.00307/) (2020)
- [A Fully Integrated Sensor-Brain-Machine Interface System for Restoring Somatosensation](https://mlsanity.com/paper/2010.15081/) (2020)
- [A System-on-Chip for Closed-loop Optogenetic Sleep Modulation](https://mlsanity.com/paper/2108.12261/) (2021)

## Built on by (most cited first, 6 of 6)

- [TinyChirp: Bird Song Recognition Using TinyML Models on Low-power Wireless Acoustic Sensors](https://mlsanity.com/paper/2407.21453/) (2024)
- [MSA-CNN: A Lightweight Multi-Scale CNN with Attention for Sleep Stage Classification](https://mlsanity.com/paper/2501.02949/) (2025)
- [Deep Learning-Powered Electrical Brain Signals Analysis: Advancing Neurological Diagnostics](https://mlsanity.com/paper/2502.17213/) (2025)
- [msf-CNN: Patch-based Multi-Stage Fusion with Convolutional Neural Networks for TinyML](https://mlsanity.com/paper/2505.11483/) (2025)
- [Transformer-based EEG Decoding: A Survey](https://mlsanity.com/paper/2507.02320/) (2025)
- [Transformer Based Model for Spatiotemporal Feature Learning in EEG Emotion Recognition](https://mlsanity.com/paper/2606.10718/) (2026)

## Abstract (arXiv)

This paper proposes a deep learning (DL) model for automatic sleep stage classification based on single-channel EEG data. The DL model features a convolutional neural network (CNN) and transformers. The model was designed to run on energy and memory-constrained devices for real-time operation with local processing. The Fpz-Cz EEG signals from a publicly available Sleep-EDF dataset are used to train and test the model. Four convolutional filter layers were used to extract features and reduce the data dimension. Then, transformers were utilized to learn the time-variant features of the data. To improve performance, we also implemented a subject specific training before the inference (i.e., prediction) stage. With the subject specific training, the F1 score was 0.91, 0.37, 0.84, 0.877, and 0.73 for wake, N1-N3, and rapid eye movement (REM) stages, respectively. The performance of the model was comparable to the state-of-the-art works with significantly greater computational costs. We tested a reduced-sized version of the proposed model on a low-cost Arduino Nano 33 BLE board and it was fully functional and accurate. In the future, a fully integrated wireless EEG sensor with edge DL will be developed for sleep research in pre-clinical and clinical experiments, such as real-time sleep modulation.