# SleepTransformer: Automatic Sleep Staging with Interpretability and Uncertainty Quantification

arXiv: https://arxiv.org/abs/2105.11043 · HTML: https://mlsanity.com/paper/2105.11043/

## Key facts

- Cited by 84 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 55 references that are in the map.
- First posted on arXiv 23 May 2021.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [DeepSleepNet: a Model for Automatic Sleep Stage Scoring based on Raw Single-Channel EEG](https://mlsanity.com/paper/1703.04046/) (2017)
- [SeqSleepNet: End-to-End Hierarchical Recurrent Neural Network for Sequence-to-Sequence Automatic Sleep Staging](https://mlsanity.com/paper/1809.10932/) (2018)
- [U-Time: A Fully Convolutional Network for Time Series Segmentation Applied to Sleep Staging](https://mlsanity.com/paper/1910.11162/) (2019)
- [XSleepNet: Multi-View Sequential Model for Automatic Sleep Staging](https://mlsanity.com/paper/2007.05492/) (2020)
- [A deep learning architecture for temporal sleep stage classification using multivariate and multimodal time series](https://mlsanity.com/paper/1707.03321/) (2017)
- [SleepEEGNet: Automated Sleep Stage Scoring with Sequence to Sequence Deep Learning Approach](https://mlsanity.com/paper/1903.02108/) (2019)
- [Joint Classification and Prediction CNN Framework for Automatic Sleep Stage Classification](https://mlsanity.com/paper/1805.06546/) (2018)
- [Neural network an1alysis of sleep stages enables efficient diagnosis of narcolepsy](https://mlsanity.com/paper/1710.02094/) (2017)
- [Dreem Open Datasets: Multi-Scored Sleep Datasets to compare Human and Automated sleep staging](https://mlsanity.com/paper/1911.03221/) (2019)
- [Towards More Accurate Automatic Sleep Staging via Deep Transfer Learning](https://mlsanity.com/paper/1907.13177/) (2019)
- [RobustSleepNet: Transfer learning for automated sleep staging at scale](https://mlsanity.com/paper/2101.02452/) (2021)
- [Mixed Neural Network Approach for Temporal Sleep Stage Classification](https://mlsanity.com/paper/1610.06421/) (2016)
- [Automatic Sleep Stage Scoring with Single-Channel EEG Using Convolutional Neural Networks](https://mlsanity.com/paper/1610.01683/) (2016)
- [Automatic sleep stage classification with deep residual networks in a mixed-cohort setting](https://mlsanity.com/paper/2008.09416/) (2020)
- [Deep Convolutional Neural Networks for Interpretable Analysis of EEG Sleep Stage Scoring](https://mlsanity.com/paper/1710.00633/) (2017)
- [Personalizing deep learning models for automatic sleep staging](https://mlsanity.com/paper/1801.02645/) (2018)
- [Deep Transfer Learning for Single-Channel Automatic Sleep Staging with Channel Mismatch](https://mlsanity.com/paper/1904.05945/) (2019)
- [TRIER: Template-Guided Neural Networks for Robust and Interpretable Sleep Stage Identification from EEG Recordings](https://mlsanity.com/paper/2009.05407/) (2020)

## Built on by (most cited first, 30 of 84)

- [CBraMod: A Criss-Cross Brain Foundation Model for EEG Decoding](https://mlsanity.com/paper/2412.07236/) (2024)
- [CSBrain: A Cross-scale Spatiotemporal Brain Foundation Model for EEG Decoding](https://mlsanity.com/paper/2506.23075/) (2025)
- [MAtt: A Manifold Attention Network for EEG Decoding](https://mlsanity.com/paper/2210.01986/) (2022)
- [Automatic Sleep Staging of EEG Signals: Recent Development, Challenges, and Future Directions](https://mlsanity.com/paper/2111.08446/) (2021)
- [L-SeqSleepNet: Whole-cycle Long Sequence Modelling for Automatic Sleep Staging](https://mlsanity.com/paper/2301.03441/) (2023)
- [EEGMamba: Bidirectional State Space Model with Mixture of Experts for EEG Multi-task Classification](https://mlsanity.com/paper/2407.20254/) (2024)
- [AdaBrain-Bench: Benchmarking Brain Foundation Models for Brain-Computer Interface Applications](https://mlsanity.com/paper/2507.09882/) (2025)
- [A Transformer-based deep neural network model for SSVEP classification](https://mlsanity.com/paper/2210.04172/) (2022)
- [Self-supervised Learning for Label-Efficient Sleep Stage Classification: A Comprehensive Evaluation](https://mlsanity.com/paper/2210.06286/) (2022)
- [SleepPPG-Net: a deep learning algorithm for robust sleep staging from continuous photoplethysmography](https://mlsanity.com/paper/2202.05735/) (2022)
- [Toward Interpretable Sleep Stage Classification Using Cross-Modal Transformers](https://mlsanity.com/paper/2208.06991/) (2022)
- [CoRe-Sleep: A Multimodal Fusion Framework for Time Series Robust to Imperfect Modalities](https://mlsanity.com/paper/2304.06485/) (2023)
- [DeeperBrain: A Neuro-Grounded EEG Foundation Model Towards Universal BCI](https://mlsanity.com/paper/2601.06134/) (2026)
- [SLEEPYLAND: trust begins with fair evaluation of automatic sleep staging models](https://mlsanity.com/paper/2506.08574/) (2025)
- [Automated Sleep Staging via Parallel Frequency-Cut Attention](https://mlsanity.com/paper/2204.03173/) (2022)
- [Do Not Sleep on Traditional Machine Learning: Simple and Interpretable Techniques Are Competitive to Deep Learning for Sleep Scoring](https://mlsanity.com/paper/2207.07753/) (2022)
- [SleePyCo: Automatic Sleep Scoring with Feature Pyramid and Contrastive Learning](https://mlsanity.com/paper/2209.09452/) (2022)
- [Fast Entropy-Based Methods of Word-Level Confidence Estimation for End-To-End Automatic Speech Recognition](https://mlsanity.com/paper/2212.08703/) (2022)
- [S4Sleep: Elucidating the design space of deep-learning-based sleep stage classification models](https://mlsanity.com/paper/2310.06715/) (2023)
- [Uncertainty Quantification in Machine Learning for Biosignal Applications -- A Review](https://mlsanity.com/paper/2312.09454/) (2023)
- [NeuroAtlas: Benchmarking Foundation Models for Clinical EEG and Brain-Computer Interfaces](https://mlsanity.com/paper/2605.14698/) (2026)
- [An Uncertainty-aware Loss Function for Training Neural Networks with Calibrated Predictions](https://mlsanity.com/paper/2110.03260/) (2021)
- [CSLP-AE: A Contrastive Split-Latent Permutation Autoencoder Framework for Zero-Shot Electroencephalography Signal Conversion](https://mlsanity.com/paper/2311.07788/) (2023)
- [Automatic Sleep Scoring from Large-scale Multi-channel Pediatric EEG](https://mlsanity.com/paper/2207.06921/) (2022)
- [PrecTime: A Deep Learning Architecture for Precise Time Series Segmentation in Industrial Manufacturing Operations](https://mlsanity.com/paper/2302.10182/) (2023)
- [BiT-MamSleep: Bidirectional Temporal Mamba for EEG Sleep Staging](https://mlsanity.com/paper/2411.01589/) (2024)
- [Domain Adaptation and Generalization on Functional Medical Images: A Systematic Survey](https://mlsanity.com/paper/2212.03176/) (2022)
- [Structure-Preserving Transformers for Sequences of SPD Matrices](https://mlsanity.com/paper/2309.07579/) (2023)
- [SleepVST: Sleep Staging from Near-Infrared Video Signals using Pre-Trained Transformers](https://mlsanity.com/paper/2404.03831/) (2024)
- [ST-USleepNet: A Spatial-Temporal Coupling Prominence Network for Multi-Channel Sleep Staging](https://mlsanity.com/paper/2408.11884/) (2024)

## Abstract (arXiv)

Background: Black-box skepticism is one of the main hindrances impeding deep-learning-based automatic sleep scoring from being used in clinical environments. Methods: Towards interpretability, this work proposes a sequence-to-sequence sleep-staging model, namely SleepTransformer. It is based on the transformer backbone and offers interpretability of the model's decisions at both the epoch and sequence level. We further propose a simple yet efficient method to quantify uncertainty in the model's decisions. The method, which is based on entropy, can serve as a metric for deferring low-confidence epochs to a human expert for further inspection. Results: Making sense of the transformer's self-attention scores for interpretability, at the epoch level, the attention scores are encoded as a heat map to highlight sleep-relevant features captured from the input EEG signal. At the sequence level, the attention scores are visualized as the influence of different neighboring epochs in an input sequence (i.e. the context) to recognition of a target epoch, mimicking the way manual scoring is done by human experts. Conclusion: Additionally, we demonstrate that SleepTransformer performs on par with existing methods on two databases of different sizes. Significance: Equipped with interpretability and the ability of uncertainty quantification, SleepTransformer holds promise for being integrated into clinical settings.