# Wearable data from subjects playing Super Mario, sitting university exams, or performing physical exercise help detect acute mood episodes via self-supervised learning

arXiv: https://arxiv.org/abs/2311.04215 · HTML: https://mlsanity.com/paper/2311.04215/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 78 references that are in the map.
- First posted on arXiv 7 Nov 2023.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization](https://mlsanity.com/paper/1603.06560/) (2016)
- [Deep learning for time series classification: a review](https://mlsanity.com/paper/1809.04356/) (2018)
- [A Transformer-based Framework for Multivariate Time Series Representation Learning](https://mlsanity.com/paper/2010.02803/) (2020)
- [Self-Supervised Representation Learning: Introduction, Advances and Challenges](https://mlsanity.com/paper/2110.09327/) (2021)
- [Are Large-scale Datasets Necessary for Self-Supervised Pre-training?](https://mlsanity.com/paper/2112.10740/) (2021)
- [Multi-Scale Convolutional Neural Networks for Time Series Classification](https://mlsanity.com/paper/1603.06995/) (2016)
- [Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects](https://mlsanity.com/paper/2306.10125/) (2023)
- [Subject-Aware Contrastive Learning for Biosignals](https://mlsanity.com/paper/2007.04871/) (2020)
- [Transformer-based Self-supervised Multimodal Representation Learning for Wearable Emotion Recognition](https://mlsanity.com/paper/2303.17611/) (2023)
- [A multimodal sensor dataset for continuous stress detection of nurses in a hospital](https://mlsanity.com/paper/2108.07689/) (2021)
- [Understanding occupants' behaviour, engagement, emotion, and comfort indoors with heterogeneous sensors and wearables](https://mlsanity.com/paper/2105.06637/) (2021)
- [ADARP: A Multi Modal Dataset for Stress and Alcohol Relapse Quantification in Real Life Setting](https://mlsanity.com/paper/2206.14568/) (2022)
- [Long-Short Ensemble Network for Bipolar Manic-Euthymic State Recognition Based on Wrist-worn Sensors](https://mlsanity.com/paper/2107.00710/) (2021)
- [The Lean Data Scientist: Recent Advances towards Overcoming the Data Bottleneck](https://mlsanity.com/paper/2211.07959/) (2022)

## Abstract (arXiv)

Personal sensing, leveraging data passively and near-continuously collected with wearables from patients in their ecological environment, is a promising paradigm to monitor mood disorders (MDs), a major determinant of worldwide disease burden. However, collecting and annotating wearable data is very resource-intensive. Studies of this kind can thus typically afford to recruit only a couple dozens of patients. This constitutes one of the major obstacles to applying modern supervised machine learning techniques to MDs detection. In this paper, we overcome this data bottleneck and advance the detection of MDs acute episode vs stable state from wearables data on the back of recent advances in self-supervised learning (SSL). This leverages unlabelled data to learn representations during pre-training, subsequently exploited for a supervised task. First, we collected open-access datasets recording with an Empatica E4 spanning different, unrelated to MD monitoring, personal sensing tasks -- from emotion recognition in Super Mario players to stress detection in undergraduates -- and devised a pre-processing pipeline performing on-/off-body detection, sleep-wake detection, segmentation, and (optionally) feature extraction. With 161 E4-recorded subjects, we introduce E4SelfLearning, the largest to date open access collection, and its pre-processing pipeline. Second, we show that SSL confidently outperforms fully-supervised pipelines using either our novel E4-tailored Transformer architecture (E4mer) or classical baseline XGBoost: 81.23% against 75.35% (E4mer) and 72.02% (XGBoost) correctly classified recording segments from 64 (half acute, half stable) patients. Lastly, we illustrate that SSL performance is strongly associated with the specific surrogate task employed for pre-training as well as with unlabelled data availability.