# U-PASS: an Uncertainty-guided deep learning Pipeline for Automated Sleep Staging

arXiv: https://arxiv.org/abs/2306.04663 · HTML: https://mlsanity.com/paper/2306.04663/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 51 references that are in the map.
- First posted on arXiv 7 Jun 2023.

## Builds on (most cited first, 20 of 20)

- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges](https://mlsanity.com/paper/2011.06225/) (2020)
- [Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics](https://mlsanity.com/paper/2009.10795/) (2020)
- [Beta Shapley: a Unified and Noise-reduced Data Valuation Framework for Machine Learning](https://mlsanity.com/paper/2110.14049/) (2021)
- [Dataset Pruning: Reducing Training Data by Examining Generalization Influence](https://mlsanity.com/paper/2205.09329/) (2022)
- [Machine Learning with a Reject Option: A survey](https://mlsanity.com/paper/2107.11277/) (2021)
- [SeqSleepNet: End-to-End Hierarchical Recurrent Neural Network for Sequence-to-Sequence Automatic Sleep Staging](https://mlsanity.com/paper/1809.10932/) (2018)
- [XSleepNet: Multi-View Sequential Model for Automatic Sleep Staging](https://mlsanity.com/paper/2007.05492/) (2020)
- [SleepTransformer: Automatic Sleep Staging with Interpretability and Uncertainty Quantification](https://mlsanity.com/paper/2105.11043/) (2021)
- [Secure and Robust Machine Learning for Healthcare: A Survey](https://mlsanity.com/paper/2001.08103/) (2020)
- [Aleatoric and Epistemic Uncertainty with Random Forests](https://mlsanity.com/paper/2001.00893/) (2020)
- [Neural network an1alysis of sleep stages enables efficient diagnosis of narcolepsy](https://mlsanity.com/paper/1710.02094/) (2017)
- [Dreem Open Datasets: Multi-Scored Sleep Datasets to compare Human and Automated sleep staging](https://mlsanity.com/paper/1911.03221/) (2019)
- [Towards More Accurate Automatic Sleep Staging via Deep Transfer Learning](https://mlsanity.com/paper/1907.13177/) (2019)
- [DeepSleepNet-Lite: A Simplified Automatic Sleep Stage Scoring Model with Uncertainty Estimates](https://mlsanity.com/paper/2108.10600/) (2021)
- [RobustSleepNet: Transfer learning for automated sleep staging at scale](https://mlsanity.com/paper/2101.02452/) (2021)
- [Data-IQ: Characterizing subgroups with heterogeneous outcomes in tabular data](https://mlsanity.com/paper/2210.13043/) (2022)
- [ADAST: Attentive Cross-domain EEG-based Sleep Staging Framework with Iterative Self-Training](https://mlsanity.com/paper/2107.04470/) (2021)
- [U-Sleep's resilience to AASM guidelines](https://mlsanity.com/paper/2209.11173/) (2022)
- [Multi-Scored Sleep Databases: How to Exploit the Multiple-Labels in Automated Sleep Scoring](https://mlsanity.com/paper/2207.01910/) (2022)

## Abstract (arXiv)

As machine learning becomes increasingly prevalent in critical fields such as healthcare, ensuring the safety and reliability of machine learning systems becomes paramount. A key component of reliability is the ability to estimate uncertainty, which enables the identification of areas of high and low confidence and helps to minimize the risk of error. In this study, we propose a machine learning pipeline called U-PASS tailored for clinical applications that incorporates uncertainty estimation at every stage of the process, including data acquisition, training, and model deployment. The training process is divided into a supervised pre-training step and a semi-supervised finetuning step. We apply our uncertainty-guided deep learning pipeline to the challenging problem of sleep staging and demonstrate that it systematically improves performance at every stage. By optimizing the training dataset, actively seeking informative samples, and deferring the most uncertain samples to an expert, we achieve an expert-level accuracy of 85% on a challenging clinical dataset of elderly sleep apnea patients, representing a significant improvement over the baseline accuracy of 75%. U-PASS represents a promising approach to incorporating uncertainty estimation into machine learning pipelines, thereby improving their reliability and unlocking their potential in clinical settings.