# Sense and Learn: Self-Supervision for Omnipresent Sensors

arXiv: https://arxiv.org/abs/2009.13233 · HTML: https://mlsanity.com/paper/2009.13233/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Sep 2020.

## Built on by (most cited first, 21 of 21)

- [Assessing the State of Self-Supervised Human Activity Recognition using Wearables](https://mlsanity.com/paper/2202.12938/) (2022)
- [Scaling Wearable Foundation Models](https://mlsanity.com/paper/2410.13638/) (2024)
- [Self-supervised Contrastive Representation Learning for Semi-supervised Time-Series Classification](https://mlsanity.com/paper/2208.06616/) (2022)
- [SelfHAR: Improving Human Activity Recognition through Self-training with Unlabeled Data](https://mlsanity.com/paper/2102.06073/) (2021)
- [Beyond Just Vision: A Review on Self-Supervised Representation Learning on Multimodal and Temporal Data](https://mlsanity.com/paper/2206.02353/) (2022)
- [ColloSSL: Collaborative Self-Supervised Learning for Human Activity Recognition](https://mlsanity.com/paper/2202.00758/) (2022)
- [COCOA: Cross Modality Contrastive Learning for Sensor Data](https://mlsanity.com/paper/2208.00467/) (2022)
- [CroSSL: Cross-modal Self-Supervised Learning for Time-series through Latent Masking](https://mlsanity.com/paper/2307.16847/) (2023)
- [Label-efficient Time Series Representation Learning: A Review](https://mlsanity.com/paper/2302.06433/) (2023)
- [Generalizable Low-Resource Activity Recognition with Diverse and Discriminative Representation Learning](https://mlsanity.com/paper/2306.04641/) (2023)
- [A Survey of Few-Shot Learning for Biomedical Time Series](https://mlsanity.com/paper/2405.02485/) (2024)
- [Self-Supervised WiFi-Based Activity Recognition](https://mlsanity.com/paper/2104.09072/) (2021)
- [Investigating Enhancements to Contrastive Predictive Coding for Human Activity Recognition](https://mlsanity.com/paper/2211.06173/) (2022)
- [Self-Supervised Human Activity Recognition with Localized Time-Frequency Contrastive Representation Learning](https://mlsanity.com/paper/2209.00990/) (2022)
- [Representation Learning for Wearable-Based Applications in the Case of Missing Data](https://mlsanity.com/paper/2401.05437/) (2024)
- [Self-Supervised Learning for WiFi CSI-Based Human Activity Recognition: A Systematic Study](https://mlsanity.com/paper/2308.02412/) (2023)
- [Self-supervised Activity Representation Learning with Incremental Data: An Empirical Study](https://mlsanity.com/paper/2305.00619/) (2023)
- [Evaluation of self-supervised pre-training for automatic infant movement classification using wearable movement sensors](https://mlsanity.com/paper/2305.09366/) (2023)
- [CIG-MAE: Cross-Modal Information-Guided Masked Autoencoder for Self-Supervised WiFi Sensing](https://mlsanity.com/paper/2512.04723/) (2025)
- [Matching Skeleton-based Activity Representations with Heterogeneous Signals for HAR](https://mlsanity.com/paper/2503.14547/) (2025)
- [Multimodal Surface EMG Hand Gesture Recognition Using Query-Based Transformers for Prosthetic Control](https://mlsanity.com/paper/2607.22779/) (2026)

## Abstract (arXiv)

Learning general-purpose representations from multisensor data produced by the omnipresent sensing systems (or IoT in general) has numerous applications in diverse use cases. Existing purely supervised end-to-end deep learning techniques depend on the availability of a massive amount of well-curated data, acquiring which is notoriously difficult but required to achieve a sufficient level of generalization on a task of interest. In this work, we leverage the self-supervised learning paradigm towards realizing the vision of continual learning from unlabeled inputs. We present a generalized framework named Sense and Learn for representation or feature learning from raw sensory data. It consists of several auxiliary tasks that can learn high-level and broadly useful features entirely from unannotated data without any human involvement in the tedious labeling process. We demonstrate the efficacy of our approach on several publicly available datasets from different domains and in various settings, including linear separability, semi-supervised or few shot learning, and transfer learning. Our methodology achieves results that are competitive with the supervised approaches and close the gap through fine-tuning a network while learning the downstream tasks in most cases. In particular, we show that the self-supervised network can be utilized as initialization to significantly boost the performance in a low-data regime with as few as 5 labeled instances per class, which is of high practical importance to real-world problems. Likewise, the learned representations with self-supervision are found to be highly transferable between related datasets, even when few labeled instances are available from the target domains. The self-learning nature of our methodology opens up exciting possibilities for on-device continual learning.