# Temporal convolutional networks and transformers for classifying the sleep stage in awake or asleep using pulse oximetry signals

arXiv: https://arxiv.org/abs/2102.03352 · HTML: https://mlsanity.com/paper/2102.03352/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 42 references that are in the map.
- First posted on arXiv 29 Jan 2021.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [Wider or Deeper: Revisiting the ResNet Model for Visual Recognition](https://mlsanity.com/paper/1611.10080/) (2016)
- [DeepSleepNet: a Model for Automatic Sleep Stage Scoring based on Raw Single-Channel EEG](https://mlsanity.com/paper/1703.04046/) (2017)
- [SeqSleepNet: End-to-End Hierarchical Recurrent Neural Network for Sequence-to-Sequence Automatic Sleep Staging](https://mlsanity.com/paper/1809.10932/) (2018)
- [SleepEEGNet: Automated Sleep Stage Scoring with Sequence to Sequence Deep Learning Approach](https://mlsanity.com/paper/1903.02108/) (2019)
- [Sleep-wake classification via quantifying heart rate variability by convolutional neural network](https://mlsanity.com/paper/1808.00142/) (2018)
- [Classifying sleep-wake stages through recurrent neural networks using pulse oximetry signals](https://mlsanity.com/paper/2008.03382/) (2020)

## Built on by (most cited first, 3 of 3)

- [A Comprehensive Survey on Applications of Transformers for Deep Learning Tasks](https://mlsanity.com/paper/2306.07303/) (2023)
- [TEMGNet: Deep Transformer-based Decoding of Upperlimb sEMG for Hand Gestures Recognition](https://mlsanity.com/paper/2109.12379/) (2021)
- [A Temporal-Spectral Fusion Transformer with Subject-Specific Adapter for Enhancing RSVP-BCI Decoding](https://mlsanity.com/paper/2401.06340/) (2024)

## Abstract (arXiv)

Sleep disorders are very widespread in the world population and suffer from a generalized underdiagnosis, given the complexity of their diagnostic methods. Therefore, there is an increasing interest in developing simpler screening methods. A pulse oximeter is an ideal device for sleep disorder screenings since it is a portable, low-cost and accessible technology. This device can provide an estimation of the heart rate (HR), which can be useful to obtain information regarding the sleep stage. In this work, we developed a network architecture with the aim of classifying the sleep stage in awake or asleep using only HR signals from a pulse oximeter. The proposed architecture has two fundamental parts. The first part has the objective of obtaining a representation of the HR by using temporal convolutional networks. Then, the obtained representation is used to feed the second part, which is based on transformers, a model built solely with attention mechanisms. Transformers are able to model the sequence, learning the transition rules between sleep stages. The performance of the proposed method was evaluated on Sleep Heart Health Study dataset, composed of 5000 healthy and pathological subjects. The dataset was split into three subsets: 2500 for training, 1250 for validating, and 1250 for testing. The overall accuracy, specificity, sensibility, and Cohen's Kappa coefficient were 90.0%, 94.9%, 78.1%, and 0.73.