# SoK: Behind the Accuracy of Complex Human Activity Recognition Using Deep Learning

arXiv: https://arxiv.org/abs/2405.00712 · HTML: https://mlsanity.com/paper/2405.00712/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 73 references that are in the map.
- First posted on arXiv 25 Apr 2024.

## Builds on (most cited first, 18 of 18)

- [Human Action Recognition from Various Data Modalities: A Review](https://mlsanity.com/paper/2012.11866/) (2020)
- [Deep Learning for Sensor-based Human Activity Recognition: Overview, Challenges and Opportunities](https://mlsanity.com/paper/2001.07416/) (2020)
- [Going Deeper into Action Recognition: A Survey](https://mlsanity.com/paper/1605.04988/) (2016)
- [Contrastive Predictive Coding for Human Activity Recognition](https://mlsanity.com/paper/2012.05333/) (2020)
- [Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances](https://mlsanity.com/paper/2111.00418/) (2021)
- [AdaFuse: Adaptive Temporal Fusion Network for Efficient Action Recognition](https://mlsanity.com/paper/2102.05775/) (2021)
- [Multimodal Co-learning: Challenges, Applications with Datasets, Recent Advances and Future Directions](https://mlsanity.com/paper/2107.13782/) (2021)
- [Human Activity Recognition using Inertial, Physiological and Environmental Sensors: a Comprehensive Survey](https://mlsanity.com/paper/2004.08821/) (2020)
- [DIVERSIFY: A General Framework for Time Series Out-of-distribution Detection and Generalization](https://mlsanity.com/paper/2308.02282/) (2023)
- [ColloSSL: Collaborative Self-Supervised Learning for Human Activity Recognition](https://mlsanity.com/paper/2202.00758/) (2022)
- [COCOA: Cross Modality Contrastive Learning for Sensor Data](https://mlsanity.com/paper/2208.00467/) (2022)
- [SWL-Adapt: An Unsupervised Domain Adaptation Model with Sample Weight Learning for Cross-User Wearable Human Activity Recognition](https://mlsanity.com/paper/2212.00724/) (2022)
- [Attend And Discriminate: Beyond the State-of-the-Art for Human Activity Recognition using Wearable Sensors](https://mlsanity.com/paper/2007.07172/) (2020)
- [Human Activity Recognition Using Tools of Convolutional Neural Networks: A State of the Art Review, Data Sets, Challenges and Future Prospects](https://mlsanity.com/paper/2202.03274/) (2022)
- [Virtual Fusion with Contrastive Learning for Single Sensor-based Activity Recognition](https://mlsanity.com/paper/2312.02185/) (2023)
- [Investigating Enhancements to Contrastive Predictive Coding for Human Activity Recognition](https://mlsanity.com/paper/2211.06173/) (2022)
- [Self-supervised Human Activity Recognition by Learning to Predict Cross-Dimensional Motion](https://mlsanity.com/paper/2010.13713/) (2020)
- [Review of Fall Detection Techniques: A Data Availability Perspective](https://mlsanity.com/paper/1605.09351/) (2016)

## Built on by (most cited first, 2 of 2)

- [Align and Adapt: Multimodal Multiview Human Activity Recognition under Arbitrary View Combinations](https://mlsanity.com/paper/2602.08755/) (2026)
- [Extending Tabular Denoising Diffusion Probabilistic Models for Time-Series Data Generation](https://mlsanity.com/paper/2604.05257/) (2026)

## Abstract (arXiv)

Human Activity Recognition (HAR) is a well-studied field with research dating back to the 1980s. Over time, HAR technologies have evolved significantly from manual feature extraction, rule-based algorithms, and simple machine learning models to powerful deep learning models, from one sensor type to a diverse array of sensing modalities. The scope has also expanded from recognising a limited set of activities to encompassing a larger variety of both simple and complex activities. However, there still exist many challenges that hinder advancement in complex activity recognition using modern deep learning methods. In this paper, we comprehensively systematise factors leading to inaccuracy in complex HAR, such as data variety and model capacity. Among many sensor types, we give more attention to wearable and camera due to their prevalence. Through this Systematisation of Knowledge (SoK) paper, readers can gain a solid understanding of the development history and existing challenges of HAR, different categorisations of activities, obstacles in deep learning-based complex HAR that impact accuracy, and potential research directions.