# Weakly Supervised Multi-Task Representation Learning for Human Activity Analysis Using Wearables

arXiv: https://arxiv.org/abs/2308.03805 · HTML: https://mlsanity.com/paper/2308.03805/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 44 references that are in the map.
- First posted on arXiv 6 Aug 2023.

## Builds on (most cited first, 13 of 13)

- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [Multi-Scale Context Aggregation by Dilated Convolutions](https://mlsanity.com/paper/1511.07122/) (2015)
- [Fully-Convolutional Siamese Networks for Object Tracking](https://mlsanity.com/paper/1606.09549/) (2016)
- [Temporal Convolutional Networks for Action Segmentation and Detection](https://mlsanity.com/paper/1611.05267/) (2016)
- [How Does Batch Normalization Help Optimization?](https://mlsanity.com/paper/1805.11604/) (2018)
- [Temporal Convolutional Networks: A Unified Approach to Action Segmentation](https://mlsanity.com/paper/1608.08242/) (2016)
- [Deep Learning for Sensor-based Activity Recognition: A Survey](https://mlsanity.com/paper/1707.03502/) (2017)
- [Deep, Convolutional, and Recurrent Models for Human Activity Recognition using Wearables](https://mlsanity.com/paper/1604.08880/) (2016)
- [Deep convolutional acoustic word embeddings using word-pair side information](https://mlsanity.com/paper/1510.01032/) (2015)
- [Understanding and Improving Recurrent Networks for Human Activity Recognition by Continuous Attention](https://mlsanity.com/paper/1810.04038/) (2018)
- [Deep Activity Recognition Models with Triaxial Accelerometers](https://mlsanity.com/paper/1511.04664/) (2015)

## Built on by (most cited first, 10 of 10)

- [Past, Present, and Future of Sensor-Based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task](https://mlsanity.com/paper/2411.14452/) (2024)
- [RAG-HAR: Retrieval Augmented Generation-based Human Activity Recognition](https://mlsanity.com/paper/2512.08984/) (2025)
- [Towards Generalizable Human Activity Recognition: A Survey](https://mlsanity.com/paper/2508.12213/) (2025)
- [A Survey of IMU Based Cross-Modal Transfer Learning in Human Activity Recognition](https://mlsanity.com/paper/2403.15444/) (2024)
- [Consistency Based Weakly Self-Supervised Learning for Human Activity Recognition with Wearables](https://mlsanity.com/paper/2408.07282/) (2024)
- [Smooth-Distill: A Self-distillation Framework for Multitask Learning with Wearable Sensor Data](https://mlsanity.com/paper/2507.00061/) (2025)
- [Reducing Label Dependency in Human Activity Recognition with Wearables: From Supervised Learning to Novel Weakly Self-Supervised Approaches](https://mlsanity.com/paper/2512.19713/) (2025)
- [Learning Unified Representations from Heterogeneous Data for Robust Heart Rate Modeling](https://mlsanity.com/paper/2508.21785/) (2025)
- [VCR: Learning Valid Contextual Representation for Incomplete Wearable Signals](https://mlsanity.com/paper/2605.18837/) (2026)
- [GenHAR: Generalizing Cross-domain Human Activity Recognition for Last-mile Delivery](https://mlsanity.com/paper/2605.22086/) (2026)

## Abstract (arXiv)

Sensor data streams from wearable devices and smart environments are widely studied in areas like human activity recognition (HAR), person identification, or health monitoring. However, most of the previous works in activity and sensor stream analysis have been focusing on one aspect of the data, e.g. only recognizing the type of the activity or only identifying the person who performed the activity. We instead propose an approach that uses a weakly supervised multi-output siamese network that learns to map the data into multiple representation spaces, where each representation space focuses on one aspect of the data. The representation vectors of the data samples are positioned in the space such that the data with the same semantic meaning in that aspect are closely located to each other. Therefore, as demonstrated with a set of experiments, the trained model can provide metrics for clustering data based on multiple aspects, allowing it to address multiple tasks simultaneously and even to outperform single task supervised methods in many situations. In addition, further experiments are presented that in more detail analyze the effect of the architecture and of using multiple tasks within this framework, that investigate the scalability of the model to include additional tasks, and that demonstrate the ability of the framework to combine data for which only partial relationship information with respect to the target tasks is available.