# SelfHAR: Improving Human Activity Recognition through Self-training with Unlabeled Data

arXiv: https://arxiv.org/abs/2102.06073 · HTML: https://mlsanity.com/paper/2102.06073/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 72 references that are in the map.
- First posted on arXiv 11 Feb 2021.

## Builds on (most cited first, 28 of 28)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [Learning Representations by Maximizing Mutual Information Across Views](https://mlsanity.com/paper/1906.00910/) (2019)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [Semi-Supervised Learning with Deep Generative Models](https://mlsanity.com/paper/1406.5298/) (2014)
- [A Survey on Deep Transfer Learning](https://mlsanity.com/paper/1808.01974/) (2018)
- [Meta-Transfer Learning for Few-Shot Learning](https://mlsanity.com/paper/1812.02391/) (2018)
- [Billion-scale semi-supervised learning for image classification](https://mlsanity.com/paper/1905.00546/) (2019)
- [Unsupervised Representation Learning by Sorting Sequences](https://mlsanity.com/paper/1708.01246/) (2017)
- [Multi-task Self-Supervised Visual Learning](https://mlsanity.com/paper/1708.07860/) (2017)
- [Data Augmentation of Wearable Sensor Data for Parkinson's Disease Monitoring using Convolutional Neural Networks](https://mlsanity.com/paper/1706.00527/) (2017)
- [Multi-task Self-Supervised Learning for Human Activity Detection](https://mlsanity.com/paper/1907.11879/) (2019)
- [Self-supervised ECG Representation Learning for Emotion Recognition](https://mlsanity.com/paper/2002.03898/) (2020)
- [Deep, Convolutional, and Recurrent Models for Human Activity Recognition using Wearables](https://mlsanity.com/paper/1604.08880/) (2016)
- [A Deeper Look at Dataset Bias](https://mlsanity.com/paper/1505.01257/) (2015)
- [Ensembles of Deep LSTM Learners for Activity Recognition using Wearables](https://mlsanity.com/paper/1703.09370/) (2017)
- [UniMiB SHAR: a new dataset for human activity recognition using acceleration data from smartphones](https://mlsanity.com/paper/1611.07688/) (2016)
- [Protecting Sensory Data against Sensitive Inferences](https://mlsanity.com/paper/1802.07802/) (2018)
- [Self-supervised transfer learning of physiological representations from free-living wearable data](https://mlsanity.com/paper/2011.12121/) (2020)
- [Sense and Learn: Self-Supervision for Omnipresent Sensors](https://mlsanity.com/paper/2009.13233/) (2020)

## Built on by (most cited first, 30 of 31)

- [Self-supervised Learning for Human Activity Recognition Using 700,000 Person-days of Wearable Data](https://mlsanity.com/paper/2206.02909/) (2022)
- [Assessing the State of Self-Supervised Human Activity Recognition using Wearables](https://mlsanity.com/paper/2202.12938/) (2022)
- [The first step is the hardest: Pitfalls of Representing and Tokenizing Temporal Data for Large Language Models](https://mlsanity.com/paper/2309.06236/) (2023)
- [ColloSSL: Collaborative Self-Supervised Learning for Human Activity Recognition](https://mlsanity.com/paper/2202.00758/) (2022)
- [IMUGPT 2.0: Language-Based Cross Modality Transfer for Sensor-Based Human Activity Recognition](https://mlsanity.com/paper/2402.01049/) (2024)
- [COCOA: Cross Modality Contrastive Learning for Sensor Data](https://mlsanity.com/paper/2208.00467/) (2022)
- [Past, Present, and Future of Sensor-Based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task](https://mlsanity.com/paper/2411.14452/) (2024)
- [Combining Public Human Activity Recognition Datasets to Mitigate Labeled Data Scarcity](https://mlsanity.com/paper/2306.13735/) (2023)
- [COMODO: Cross-Modal Video-to-IMU Distillation for Efficient Egocentric Human Activity Recognition](https://mlsanity.com/paper/2503.07259/) (2025)
- [SemiPFL: Personalized Semi-Supervised Federated Learning Framework for Edge Intelligence](https://mlsanity.com/paper/2203.08176/) (2022)
- [Large Language Model-Guided Semantic Alignment for Human Activity Recognition](https://mlsanity.com/paper/2410.00003/) (2024)
- [Large Scale Time-Series Representation Learning via Simultaneous Low and High Frequency Feature Bootstrapping](https://mlsanity.com/paper/2204.11291/) (2022)
- [Using Self-supervised Learning Can Improve Model Fairness](https://mlsanity.com/paper/2406.02361/) (2024)
- [Leveraging Large Language Models for Explainable Activity Recognition in Smart Homes: A Critical Evaluation](https://mlsanity.com/paper/2503.16622/) (2025)
- [DAPPER: Label-Free Performance Estimation after Personalization for Heterogeneous Mobile Sensing](https://mlsanity.com/paper/2111.11053/) (2021)
- [Self-supervised learning-based general laboratory progress pretrained model for cardiovascular event detection](https://mlsanity.com/paper/2303.06980/) (2023)
- [ConvBoost: Boosting ConvNets for Sensor-based Activity Recognition](https://mlsanity.com/paper/2305.13541/) (2023)
- [Balancing Continual Learning and Fine-tuning for Human Activity Recognition](https://mlsanity.com/paper/2401.02255/) (2024)
- [Transformer-Based Behavioral Representation Learning Enables Transfer Learning for Mobile Sensing in Small Datasets](https://mlsanity.com/paper/2107.06097/) (2021)
- [IMG2IMU: Translating Knowledge from Large-Scale Images to IMU Sensing Applications](https://mlsanity.com/paper/2209.00945/) (2022)
- [Don't freeze: Finetune encoders for better Self-Supervised HAR](https://mlsanity.com/paper/2307.01168/) (2023)
- [A Comprehensive Review of Automated Data Annotation Techniques in Human Activity Recognition](https://mlsanity.com/paper/2307.05988/) (2023)
- [StatioCL: Contrastive Learning for Time Series via Non-Stationary and Temporal Contrast](https://mlsanity.com/paper/2410.10048/) (2024)
- [Guided-GAN: Adversarial Representation Learning for Activity Recognition with Wearables](https://mlsanity.com/paper/2110.05732/) (2021)
- [Evaluation of self-supervised pre-training for automatic infant movement classification using wearable movement sensors](https://mlsanity.com/paper/2305.09366/) (2023)
- [Game of LLMs: Discovering Structural Constructs in Activities using Large Language Models](https://mlsanity.com/paper/2406.13777/) (2024)
- [Experience Paper: Adopting Activity Recognition in On-demand Food Delivery Business](https://mlsanity.com/paper/2509.24303/) (2025)
- [AdaNODEs: Test Time Adaptation for Time Series Forecasting Using Neural ODEs](https://mlsanity.com/paper/2601.12893/) (2026)
- [Consistency Training of Multi-exit Architectures for Sensor Data](https://mlsanity.com/paper/2109.13192/) (2021)
- [FM-FoG: A Real-Time Foundation Model-based Wearable System for Freezing-of-Gait Mitigation](https://mlsanity.com/paper/2509.24176/) (2025)

## Abstract (arXiv)

Machine learning and deep learning have shown great promise in mobile sensing applications, including Human Activity Recognition. However, the performance of such models in real-world settings largely depends on the availability of large datasets that captures diverse behaviors. Recently, studies in computer vision and natural language processing have shown that leveraging massive amounts of unlabeled data enables performance on par with state-of-the-art supervised models. In this work, we present SelfHAR, a semi-supervised model that effectively learns to leverage unlabeled mobile sensing datasets to complement small labeled datasets. Our approach combines teacher-student self-training, which distills the knowledge of unlabeled and labeled datasets while allowing for data augmentation, and multi-task self-supervision, which learns robust signal-level representations by predicting distorted versions of the input. We evaluated SelfHAR on various HAR datasets and showed state-of-the-art performance over supervised and previous semi-supervised approaches, with up to 12% increase in F1 score using the same number of model parameters at inference. Furthermore, SelfHAR is data-efficient, reaching similar performance using up to 10 times less labeled data compared to supervised approaches. Our work not only achieves state-of-the-art performance in a diverse set of HAR datasets, but also sheds light on how pre-training tasks may affect downstream performance.