# Self-supervised transfer learning of physiological representations from free-living wearable data

arXiv: https://arxiv.org/abs/2011.12121 · HTML: https://mlsanity.com/paper/2011.12121/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Nov 2020.

## Built on by (most cited first, 27 of 27)

- [Large-scale Training of Foundation Models for Wearable Biosignals](https://mlsanity.com/paper/2312.05409/) (2023)
- [PaPaGei: Open Foundation Models for Optical Physiological Signals](https://mlsanity.com/paper/2410.20542/) (2024)
- [Scaling Wearable Foundation Models](https://mlsanity.com/paper/2410.13638/) (2024)
- [The first step is the hardest: Pitfalls of Representing and Tokenizing Temporal Data for Large Language Models](https://mlsanity.com/paper/2309.06236/) (2023)
- [Toward Foundation Model for Multivariate Wearable Sensing of Physiological Signals](https://mlsanity.com/paper/2412.09758/) (2024)
- [SelfHAR: Improving Human Activity Recognition through Self-training with Unlabeled Data](https://mlsanity.com/paper/2102.06073/) (2021)
- [UniMTS: Unified Pre-training for Motion Time Series](https://mlsanity.com/paper/2410.19818/) (2024)
- [LSM-2: Learning from Incomplete Wearable Sensor Data](https://mlsanity.com/paper/2506.05321/) (2025)
- [CroSSL: Cross-modal Self-Supervised Learning for Time-series through Latent Masking](https://mlsanity.com/paper/2307.16847/) (2023)
- [Wearable Accelerometer Foundation Models for Health via Knowledge Distillation](https://mlsanity.com/paper/2412.11276/) (2024)
- [From Personalized Medicine to Population Health: A Survey of mHealth Sensing Techniques](https://mlsanity.com/paper/2107.00948/) (2021)
- [GlucoFM: A Dual-Stream Foundation Model for Continuous Glucose Monitoring](https://mlsanity.com/paper/2605.30865/) (2026)
- [(Un)fair devices: Moving beyond AI accuracy in personal sensing](https://mlsanity.com/paper/2303.15585/) (2023)
- [MoPFormer: Motion-Primitive Transformer for Wearable-Sensor Activity Recognition](https://mlsanity.com/paper/2505.20744/) (2025)
- [Using Self-supervised Learning Can Improve Model Fairness](https://mlsanity.com/paper/2406.02361/) (2024)
- [SympCam: Remote Optical Measurement of Sympathetic Arousal](https://mlsanity.com/paper/2410.20552/) (2024)
- [More to Less (M2L): Enhanced Health Recognition in the Wild with Reduced Modality of Wearable Sensors](https://mlsanity.com/paper/2202.08267/) (2022)
- [PhysioMTL: Personalizing Physiological Patterns using Optimal Transport Multi-Task Regression](https://mlsanity.com/paper/2203.12595/) (2022)
- [Longitudinal cardio-respiratory fitness prediction through wearables in free-living environments](https://mlsanity.com/paper/2205.03116/) (2022)
- [Self-supervised learning-based general laboratory progress pretrained model for cardiovascular event detection](https://mlsanity.com/paper/2303.06980/) (2023)
- [Uncovering Bias in Personal Informatics](https://mlsanity.com/paper/2303.15592/) (2023)
- [UDAMA: Unsupervised Domain Adaptation through Multi-discriminator Adversarial Training with Noisy Labels Improves Cardio-fitness Prediction](https://mlsanity.com/paper/2307.16651/) (2023)
- [Daily Physical Activity Monitoring -- Adaptive Learning from Multi-source Motion Sensor Data](https://mlsanity.com/paper/2405.16395/) (2024)
- [Evaluating Fairness in Self-supervised and Supervised Models for Sequential Data](https://mlsanity.com/paper/2401.01640/) (2024)
- [Turning Silver into Gold: Domain Adaptation with Noisy Labels for Wearable Cardio-Respiratory Fitness Prediction](https://mlsanity.com/paper/2211.10475/) (2022)
- [Personalization of Stress Mobile Sensing using Self-Supervised Learning](https://mlsanity.com/paper/2308.02731/) (2023)
- [Emergent Symbolic Structure in Health Foundation Models: Extraction, Alignment, and Cross-Modal Transfer](https://mlsanity.com/paper/2605.07407/) (2026)

## Abstract (arXiv)

Wearable devices such as smartwatches are becoming increasingly popular tools for objectively monitoring physical activity in free-living conditions. To date, research has primarily focused on the purely supervised task of human activity recognition, demonstrating limited success in inferring high-level health outcomes from low-level signals. Here, we present a novel self-supervised representation learning method using activity and heart rate (HR) signals without semantic labels. With a deep neural network, we set HR responses as the supervisory signal for the activity data, leveraging their underlying physiological relationship. In addition, we propose a custom quantile loss function that accounts for the long-tailed HR distribution present in the general population. We evaluate our model in the largest free-living combined-sensing dataset (comprising >280k hours of wrist accelerometer & wearable ECG data). Our contributions are two-fold: i) the pre-training task creates a model that can accurately forecast HR based only on cheap activity sensors, and ii) we leverage the information captured through this task by proposing a simple method to aggregate the learnt latent representations (embeddings) from the window-level to user-level. Notably, we show that the embeddings can generalize in various downstream tasks through transfer learning with linear classifiers, capturing physiologically meaningful, personalized information. For instance, they can be used to predict variables associated with individuals' health, fitness and demographic characteristics, outperforming unsupervised autoencoders and common bio-markers. Overall, we propose the first multimodal self-supervised method for behavioral and physiological data with implications for large-scale health and lifestyle monitoring.