# TransSleep: Transitioning-aware Attention-based Deep Neural Network for Sleep Staging

arXiv: https://arxiv.org/abs/2203.12590 · HTML: https://mlsanity.com/paper/2203.12590/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 54 references that are in the map.
- First posted on arXiv 22 Mar 2022.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Network In Network](https://mlsanity.com/paper/1312.4400/) (2013)
- [DeepSleepNet: a Model for Automatic Sleep Stage Scoring based on Raw Single-Channel EEG](https://mlsanity.com/paper/1703.04046/) (2017)
- [SeqSleepNet: End-to-End Hierarchical Recurrent Neural Network for Sequence-to-Sequence Automatic Sleep Staging](https://mlsanity.com/paper/1809.10932/) (2018)
- [U-Time: A Fully Convolutional Network for Time Series Segmentation Applied to Sleep Staging](https://mlsanity.com/paper/1910.11162/) (2019)
- [XSleepNet: Multi-View Sequential Model for Automatic Sleep Staging](https://mlsanity.com/paper/2007.05492/) (2020)
- [Joint Classification and Prediction CNN Framework for Automatic Sleep Stage Classification](https://mlsanity.com/paper/1805.06546/) (2018)
- [Deep Learning on Small Datasets without Pre-Training using Cosine Loss](https://mlsanity.com/paper/1901.09054/) (2019)
- [Evolutionary Neural Architecture Search for Image Restoration](https://mlsanity.com/paper/1812.05866/) (2018)
- [Multi-Scale Neural network for EEG Representation Learning in BCI](https://mlsanity.com/paper/2003.02657/) (2020)

## Built on by (most cited first, 8 of 8)

- [L-SeqSleepNet: Whole-cycle Long Sequence Modelling for Automatic Sleep Staging](https://mlsanity.com/paper/2301.03441/) (2023)
- [Toward Foundational Model for Sleep Analysis Using a Multimodal Hybrid Self-Supervised Learning Framework](https://mlsanity.com/paper/2502.17481/) (2025)
- [SleepEGAN: A GAN-enhanced Ensemble Deep Learning Model for Imbalanced Classification of Sleep Stages](https://mlsanity.com/paper/2307.05362/) (2023)
- [EEG-MedRAG: Enhancing EEG-based Clinical Decision-Making via Hierarchical Hypergraph Retrieval-Augmented Generation](https://mlsanity.com/paper/2508.13735/) (2025)
- [MEASURE: Multi-scale Minimal Sufficient Representation Learning for Domain Generalization in Sleep Staging](https://mlsanity.com/paper/2510.12070/) (2025)
- [Local-Global Temporal Fusion Network with an Attention Mechanism for Multiple and Multiclass Arrhythmia Classification](https://mlsanity.com/paper/2308.02416/) (2023)
- [Redundant feature screening method for human activity recognition based on attention purification mechanism](https://mlsanity.com/paper/2503.23537/) (2025)
- [Resource Efficient Sleep Staging via Multi-Level Masking and Prompt Learning](https://mlsanity.com/paper/2511.06785/) (2025)

## Abstract (arXiv)

Sleep staging is essential for sleep assessment and plays a vital role as a health indicator. Many recent studies have devised various machine learning as well as deep learning architectures for sleep staging. However, two key challenges hinder the practical use of these architectures: effectively capturing salient waveforms in sleep signals and correctly classifying confusing stages in transitioning epochs. In this study, we propose a novel deep neural network structure, TransSleep, that captures distinctive local temporal patterns and distinguishes confusing stages using two auxiliary tasks. In particular, TransSleep adopts an attention-based multi-scale feature extractor module to capture salient waveforms; a stage-confusion estimator module with a novel auxiliary task, epoch-level stage classification, to estimate confidence scores for identifying confusing stages; and a context encoder module with the other novel auxiliary task, stage-transition detection, to represent contextual relationships across neighboring epochs. Results show that TransSleep achieves promising performance in automatic sleep staging. The validity of TransSleep is demonstrated by its state-of-the-art performance on two publicly available datasets, Sleep-EDF and MASS. Furthermore, we performed ablations to analyze our results from different perspectives. Based on our overall results, we believe that TransSleep has immense potential to provide new insights into deep learning-based sleep staging.