# A deep learning architecture for temporal sleep stage classification using multivariate and multimodal time series

arXiv: https://arxiv.org/abs/1707.03321 · HTML: https://mlsanity.com/paper/1707.03321/

## Key facts

- Cited by 58 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 5 Jul 2017.

## Built on by (most cited first, 30 of 58)

- [BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data](https://mlsanity.com/paper/2101.12037/) (2021)
- [SleepTransformer: Automatic Sleep Staging with Interpretability and Uncertainty Quantification](https://mlsanity.com/paper/2105.11043/) (2021)
- [Automatic Sleep Staging of EEG Signals: Recent Development, Challenges, and Future Directions](https://mlsanity.com/paper/2111.08446/) (2021)
- [Multi-View Spatial-Temporal Graph Convolutional Networks with Domain Generalization for Sleep Stage Classification](https://mlsanity.com/paper/2109.01824/) (2021)
- [Data augmentation for learning predictive models on EEG: a systematic comparison](https://mlsanity.com/paper/2206.14483/) (2022)
- [SalientSleepNet: Multimodal Salient Wave Detection Network for Sleep Staging](https://mlsanity.com/paper/2105.13864/) (2021)
- [DeepSleepNet-Lite: A Simplified Automatic Sleep Stage Scoring Model with Uncertainty Estimates](https://mlsanity.com/paper/2108.10600/) (2021)
- [CADDA: Class-wise Automatic Differentiable Data Augmentation for EEG Signals](https://mlsanity.com/paper/2106.13695/) (2021)
- [2021 BEETL Competition: Advancing Transfer Learning for Subject Independence & Heterogenous EEG Data Sets](https://mlsanity.com/paper/2202.12950/) (2022)
- [Robust learning from corrupted EEG with dynamic spatial filtering](https://mlsanity.com/paper/2105.12916/) (2021)
- [U-Sleep's resilience to AASM guidelines](https://mlsanity.com/paper/2209.11173/) (2022)
- [Towards Robust Multimodal Physiological Foundation Models: Handling Arbitrary Missing Modalities](https://mlsanity.com/paper/2504.19596/) (2025)
- [HetMAML: Task-Heterogeneous Model-Agnostic Meta-Learning for Few-Shot Learning Across Modalities](https://mlsanity.com/paper/2105.07889/) (2021)
- [Evaluating the structure of cognitive tasks with transfer learning](https://mlsanity.com/paper/2308.02408/) (2023)
- [Do Not Sleep on Traditional Machine Learning: Simple and Interpretable Techniques Are Competitive to Deep Learning for Sleep Scoring](https://mlsanity.com/paper/2207.07753/) (2022)
- [SleePyCo: Automatic Sleep Scoring with Feature Pyramid and Contrastive Learning](https://mlsanity.com/paper/2209.09452/) (2022)
- [Deep invariant networks with differentiable augmentation layers](https://mlsanity.com/paper/2202.02142/) (2022)
- [MSED: a multi-modal sleep event detection model for clinical sleep analysis](https://mlsanity.com/paper/2101.02530/) (2021)
- [Learning Product Graphs from Spectral Templates](https://mlsanity.com/paper/2211.02893/) (2022)
- [ProductGraphSleepNet: Sleep Staging using Product Spatio-Temporal Graph Learning with Attentive Temporal Aggregation](https://mlsanity.com/paper/2212.04881/) (2022)
- [Latent Alignment with Deep Set EEG Decoders](https://mlsanity.com/paper/2311.17968/) (2023)
- [Mining of Switching Sparse Networks for Missing Value Imputation in Multivariate Time Series](https://mlsanity.com/paper/2409.09930/) (2024)
- [SOM-CPC: Unsupervised Contrastive Learning with Self-Organizing Maps for Structured Representations of High-Rate Time Series](https://mlsanity.com/paper/2205.15875/) (2022)
- [Multi-Scored Sleep Databases: How to Exploit the Multiple-Labels in Automated Sleep Scoring](https://mlsanity.com/paper/2207.01910/) (2022)
- [A multi-level interpretable sleep stage scoring system by infusing experts' knowledge into a deep network architecture](https://mlsanity.com/paper/2207.04585/) (2022)
- [Multi-Source and Test-Time Domain Adaptation on Multivariate Signals using Spatio-Temporal Monge Alignment](https://mlsanity.com/paper/2407.14303/) (2024)
- [A novel dual-stream time-frequency contrastive pretext tasks framework for sleep stage classification](https://mlsanity.com/paper/2312.09623/) (2023)
- [MSA-CNN: A Lightweight Multi-Scale CNN with Attention for Sleep Stage Classification](https://mlsanity.com/paper/2501.02949/) (2025)
- [Deep Learning-Powered Electrical Brain Signals Analysis: Advancing Neurological Diagnostics](https://mlsanity.com/paper/2502.17213/) (2025)
- [Leveraging Generic Time Series Foundation Models for EEG Classification](https://mlsanity.com/paper/2510.27522/) (2025)

## Abstract (arXiv)

Sleep stage classification constitutes an important preliminary exam in the diagnosis of sleep disorders. It is traditionally performed by a sleep expert who assigns to each 30s of signal a sleep stage, based on the visual inspection of signals such as electroencephalograms (EEG), electrooculograms (EOG), electrocardiograms (ECG) and electromyograms (EMG). We introduce here the first deep learning approach for sleep stage classification that learns end-to-end without computing spectrograms or extracting hand-crafted features, that exploits all multivariate and multimodal Polysomnography (PSG) signals (EEG, EMG and EOG), and that can exploit the temporal context of each 30s window of data. For each modality the first layer learns linear spatial filters that exploit the array of sensors to increase the signal-to-noise ratio, and the last layer feeds the learnt representation to a softmax classifier. Our model is compared to alternative automatic approaches based on convolutional networks or decisions trees. Results obtained on 61 publicly available PSG records with up to 20 EEG channels demonstrate that our network architecture yields state-of-the-art performance. Our study reveals a number of insights on the spatio-temporal distribution of the signal of interest: a good trade-off for optimal classification performance measured with balanced accuracy is to use 6 EEG with 2 EOG (left and right) and 3 EMG chin channels. Also exploiting one minute of data before and after each data segment offers the strongest improvement when a limited number of channels is available. As sleep experts, our system exploits the multivariate and multimodal nature of PSG signals in order to deliver state-of-the-art classification performance with a small computational cost.