# Dreem Open Datasets: Multi-Scored Sleep Datasets to compare Human and Automated sleep staging

arXiv: https://arxiv.org/abs/1911.03221 · HTML: https://mlsanity.com/paper/1911.03221/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2019.

## Built on by (most cited first, 30 of 33)

- [SleepTransformer: Automatic Sleep Staging with Interpretability and Uncertainty Quantification](https://mlsanity.com/paper/2105.11043/) (2021)
- [Automatic Sleep Staging of EEG Signals: Recent Development, Challenges, and Future Directions](https://mlsanity.com/paper/2111.08446/) (2021)
- [Modeling Multivariate Biosignals With Graph Neural Networks and Structured State Space Models](https://mlsanity.com/paper/2211.11176/) (2022)
- [Self-supervised Contrastive Learning for EEG-based Sleep Staging](https://mlsanity.com/paper/2109.07839/) (2021)
- [RobustSleepNet: Transfer learning for automated sleep staging at scale](https://mlsanity.com/paper/2101.02452/) (2021)
- [L-SeqSleepNet: Whole-cycle Long Sequence Modelling for Automatic Sleep Staging](https://mlsanity.com/paper/2301.03441/) (2023)
- [Robust learning from corrupted EEG with dynamic spatial filtering](https://mlsanity.com/paper/2105.12916/) (2021)
- [SPDIM: Source-Free Unsupervised Conditional and Label Shift Adaptation in EEG](https://mlsanity.com/paper/2411.07249/) (2024)
- [Do Not Sleep on Traditional Machine Learning: Simple and Interpretable Techniques Are Competitive to Deep Learning for Sleep Scoring](https://mlsanity.com/paper/2207.07753/) (2022)
- [NeuroAtlas: Benchmarking Foundation Models for Clinical EEG and Brain-Computer Interfaces](https://mlsanity.com/paper/2605.14698/) (2026)
- [Multi-Scored Sleep Databases: How to Exploit the Multiple-Labels in Automated Sleep Scoring](https://mlsanity.com/paper/2207.01910/) (2022)
- [Contaminated Multivariate Time-Series Anomaly Detection with Spatio-Temporal Graph Conditional Diffusion Models](https://mlsanity.com/paper/2308.12563/) (2023)
- [A Survey of Spatio-Temporal EEG data Analysis: from Models to Applications](https://mlsanity.com/paper/2410.08224/) (2024)
- [Deep Learning-Powered Electrical Brain Signals Analysis: Advancing Neurological Diagnostics](https://mlsanity.com/paper/2502.17213/) (2025)
- [SleepMaMi: A Universal Sleep Foundation Model for Integrating Macro- and Micro-structures](https://mlsanity.com/paper/2602.07628/) (2026)
- [SERF: Interpretable Sleep Staging using Embeddings, Rules, and Features](https://mlsanity.com/paper/2209.11174/) (2022)
- [aSAGA: Automatic Sleep Analysis with Gray Areas](https://mlsanity.com/paper/2310.02032/) (2023)
- [Domain Invariant Representation Learning and Sleep Dynamics Modeling for Automatic Sleep Staging](https://mlsanity.com/paper/2312.03196/) (2023)
- [NAPS: Attention-Based Fusion of Heterogeneous Physiological Signals](https://mlsanity.com/paper/2511.03488/) (2025)
- [AnySleep: a channel-agnostic deep learning system for high-resolution sleep staging in multi-center cohorts](https://mlsanity.com/paper/2512.14461/) (2025)
- [Next-Token Prediction Learns Generalisable Representations of Sleep Physiology](https://mlsanity.com/paper/2606.09605/) (2026)
- [Performance and utility trade-off in interpretable sleep staging](https://mlsanity.com/paper/2211.03282/) (2022)
- [U-PASS: an Uncertainty-guided deep learning Pipeline for Automated Sleep Staging](https://mlsanity.com/paper/2306.04663/) (2023)
- [Comparison analysis between standard polysomnographic data and in-ear-EEG signals: A preliminary study](https://mlsanity.com/paper/2401.10107/) (2024)
- [MorpheusNet: Resource efficient sleep stage classifier for embedded on-line systems](https://mlsanity.com/paper/2401.10284/) (2024)
- [Data-Efficient Sleep Staging with Synthetic Time Series Pretraining](https://mlsanity.com/paper/2403.08592/) (2024)
- [Alzheimer's disease detection in PSG signals](https://mlsanity.com/paper/2404.03549/) (2024)
- [Automatic Classification of Sleep Stages from EEG Signals Using Riemannian Metrics and Transformer Networks](https://mlsanity.com/paper/2410.19819/) (2024)
- [From Sleep Staging to Spindle Detection: A Case Study on End-to-End Automated Sleep Analysis](https://mlsanity.com/paper/2505.05371/) (2025)
- [A Systematic Evaluation of Self-Supervised Learning for Label-Efficient Sleep Staging with Wearable EEG](https://mlsanity.com/paper/2510.07960/) (2025)

## Abstract (arXiv)

Sleep stage classification constitutes an important element of sleep disorder diagnosis. It relies on the visual inspection of polysomnography records by trained sleep technologists. Automated approaches have been designed to alleviate this resource-intensive task. However, such approaches are usually compared to a single human scorer annotation despite an inter-rater agreement of about 85 % only. The present study introduces two publicly-available datasets, DOD-H including 25 healthy volunteers and DOD-O including 55 patients suffering from obstructive sleep apnea (OSA). Both datasets have been scored by 5 sleep technologists from different sleep centers. We developed a framework to compare automated approaches to a consensus of multiple human scorers. Using this framework, we benchmarked and compared the main literature approaches. We also developed and benchmarked a new deep learning method, SimpleSleepNet, inspired by current state-of-the-art. We demonstrated that many methods can reach human-level performance on both datasets. SimpleSleepNet achieved an F1 of 89.9 % vs 86.8 % on average for human scorers on DOD-H, and an F1 of 88.3 % vs 84.8 % on DOD-O. Our study highlights that using state-of-the-art automated sleep staging outperforms human scorers performance for healthy volunteers and patients suffering from OSA. Consideration could be made to use automated approaches in the clinical setting.