# Temporal Feature Alignment in Contrastive Self-Supervised Learning for Human Activity Recognition

arXiv: https://arxiv.org/abs/2210.03382 · HTML: https://mlsanity.com/paper/2210.03382/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 43 references that are in the map.
- First posted on arXiv 7 Oct 2022.

## Builds on (most cited first, 22 of 22)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [Large Batch Training of Convolutional Networks](https://mlsanity.com/paper/1708.03888/) (2017)
- [Soft-DTW: a Differentiable Loss Function for Time-Series](https://mlsanity.com/paper/1703.01541/) (2017)
- [An End-to-End Spatio-Temporal Attention Model for Human Action Recognition from Skeleton Data](https://mlsanity.com/paper/1611.06067/) (2016)
- [Deep Learning for Sensor-based Human Activity Recognition: Overview, Challenges and Opportunities](https://mlsanity.com/paper/2001.07416/) (2020)
- [Co-occurrence Feature Learning from Skeleton Data for Action Recognition and Detection with Hierarchical Aggregation](https://mlsanity.com/paper/1804.06055/) (2018)
- [Multi-task Self-Supervised Learning for Human Activity Detection](https://mlsanity.com/paper/1907.11879/) (2019)
- [Deep, Convolutional, and Recurrent Models for Human Activity Recognition using Wearables](https://mlsanity.com/paper/1604.08880/) (2016)
- [Contrastive Predictive Coding for Human Activity Recognition](https://mlsanity.com/paper/2012.05333/) (2020)
- [Learning by Aligning Videos in Time](https://mlsanity.com/paper/2103.17260/) (2021)
- [Semantics-aware Adaptive Knowledge Distillation for Sensor-to-Vision Action Recognition](https://mlsanity.com/paper/2009.00210/) (2020)
- [HAMLET: A Hierarchical Multimodal Attention-based Human Activity Recognition Algorithm](https://mlsanity.com/paper/2008.01148/) (2020)
- [Understanding and Improving Recurrent Networks for Human Activity Recognition by Continuous Attention](https://mlsanity.com/paper/1810.04038/) (2018)
- [Human Activity Recognition from Wearable Sensor Data Using Self-Attention](https://mlsanity.com/paper/2003.09018/) (2020)
- [Deep Residual Bidir-LSTM for Human Activity Recognition Using Wearable Sensors](https://mlsanity.com/paper/1708.08989/) (2017)
- [Gimme Signals: Discriminative signal encoding for multimodal activity recognition](https://mlsanity.com/paper/2003.06156/) (2020)

## Built on by (most cited first, 4 of 4)

- [A Survey on Multimodal Wearable Sensor-based Human Action Recognition](https://mlsanity.com/paper/2404.15349/) (2024)
- [Neural Latent Aligner: Cross-trial Alignment for Learning Representations of Complex, Naturalistic Neural Data](https://mlsanity.com/paper/2308.06443/) (2023)
- [Multimodal Contrastive Learning with Hard Negative Sampling for Human Activity Recognition](https://mlsanity.com/paper/2309.01262/) (2023)
- [Enhancing IMU-Based Online Handwriting Recognition via Contrastive Learning with Zero Inference Overhead](https://mlsanity.com/paper/2602.07049/) (2026)

## Abstract (arXiv)

Automated Human Activity Recognition has long been a problem of great interest in human-centered and ubiquitous computing. In the last years, a plethora of supervised learning algorithms based on deep neural networks has been suggested to address this problem using various modalities. While every modality has its own limitations, there is one common challenge. Namely, supervised learning requires vast amounts of annotated data which is practically hard to collect. In this paper, we benefit from the self-supervised learning paradigm (SSL) that is typically used to learn deep feature representations from unlabeled data. Moreover, we upgrade a contrastive SSL framework, namely SimCLR, widely used in various applications by introducing a temporal feature alignment procedure for Human Activity Recognition. Specifically, we propose integrating a dynamic time warping (DTW) algorithm in a latent space to force features to be aligned in a temporal dimension. Extensive experiments have been conducted for the unimodal scenario with inertial modality as well as in multimodal settings using inertial and skeleton data. According to the obtained results, the proposed approach has a great potential in learning robust feature representations compared to the recent SSL baselines, and clearly outperforms supervised models in semi-supervised learning. The code for the unimodal case is available via the following link: https://github.com/bulatkh/csshar_tfa.