# Multi-Scored Sleep Databases: How to Exploit the Multiple-Labels in Automated Sleep Scoring

arXiv: https://arxiv.org/abs/2207.01910 · HTML: https://mlsanity.com/paper/2207.01910/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 44 references that are in the map.
- First posted on arXiv 5 Jul 2022.

## Builds on (most cited first, 17 of 17)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [DeepSleepNet: a Model for Automatic Sleep Stage Scoring based on Raw Single-Channel EEG](https://mlsanity.com/paper/1703.04046/) (2017)
- [SeqSleepNet: End-to-End Hierarchical Recurrent Neural Network for Sequence-to-Sequence Automatic Sleep Staging](https://mlsanity.com/paper/1809.10932/) (2018)
- [XSleepNet: Multi-View Sequential Model for Automatic Sleep Staging](https://mlsanity.com/paper/2007.05492/) (2020)
- [A deep learning architecture for temporal sleep stage classification using multivariate and multimodal time series](https://mlsanity.com/paper/1707.03321/) (2017)
- [SleepEEGNet: Automated Sleep Stage Scoring with Sequence to Sequence Deep Learning Approach](https://mlsanity.com/paper/1903.02108/) (2019)
- [Neural network an1alysis of sleep stages enables efficient diagnosis of narcolepsy](https://mlsanity.com/paper/1710.02094/) (2017)
- [Dreem Open Datasets: Multi-Scored Sleep Datasets to compare Human and Automated sleep staging](https://mlsanity.com/paper/1911.03221/) (2019)
- [DeepSleepNet-Lite: A Simplified Automatic Sleep Stage Scoring Model with Uncertainty Estimates](https://mlsanity.com/paper/2108.10600/) (2021)
- [Mixed Neural Network Approach for Temporal Sleep Stage Classification](https://mlsanity.com/paper/1610.06421/) (2016)
- [Automatic sleep stage classification with deep residual networks in a mixed-cohort setting](https://mlsanity.com/paper/2008.09416/) (2020)
- [Deep Convolutional Neural Networks for Interpretable Analysis of EEG Sleep Stage Scoring](https://mlsanity.com/paper/1710.00633/) (2017)

## Built on by (most cited first, 4 of 4)

- [SLEEPYLAND: trust begins with fair evaluation of automatic sleep staging models](https://mlsanity.com/paper/2506.08574/) (2025)
- [A Deep Generative Model for Five-Class Sleep Staging with Arbitrary Sensor Input](https://mlsanity.com/paper/2408.15253/) (2024)
- [U-PASS: an Uncertainty-guided deep learning Pipeline for Automated Sleep Staging](https://mlsanity.com/paper/2306.04663/) (2023)
- [Comparison analysis between standard polysomnographic data and in-ear-EEG signals: A preliminary study](https://mlsanity.com/paper/2401.10107/) (2024)

## Abstract (arXiv)

Study Objectives: Inter-scorer variability in scoring polysomnograms is a well-known problem. Most of the existing automated sleep scoring systems are trained using labels annotated by a single scorer, whose subjective evaluation is transferred to the model. When annotations from two or more scorers are available, the scoring models are usually trained on the scorer consensus. The averaged scorer's subjectivity is transferred into the model, losing information about the internal variability among different scorers. In this study, we aim to insert the multiple-knowledge of the different physicians into the training procedure. The goal is to optimize a model training, exploiting the full information that can be extracted from the consensus of a group of scorers. Methods: We train two lightweight deep learning based models on three different multi-scored databases. We exploit the label smoothing technique together with a soft-consensus (LSSC) distribution to insert the multiple-knowledge in the training procedure of the model. We introduce the averaged cosine similarity metric (ACS) to quantify the similarity between the hypnodensity-graph generated by the models with-LSSC and the hypnodensity-graph generated by the scorer consensus. Results: The performance of the models improves on all the databases when we train the models with our LSSC. We found an increase in ACS (up to 6.4%) between the hypnodensity-graph generated by the models trained with-LSSC and the hypnodensity-graph generated by the consensus. Conclusion: Our approach definitely enables a model to better adapt to the consensus of the group of scorers. Future work will focus on further investigations on different scoring architectures and hopefully large-scale-heterogeneous multi-scored datasets.