mlsanityData to 7 Oct 2026

A Comprehensive Methodological Survey of Human Activity Recognition Across Divers Data Modalities

arXiv 2409.09678 ↗First posted 15 Sep 2024cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Human Activity Recognition (HAR) systems aim to understand human behaviour and assign a label to each action, attracting significant attention in computer vision due to their wide range of applications. HAR can leverage various data modalities, such as RGB images and video, skeleton, depth, infrared, point cloud, event stream, audio, acceleration, and radar signals. Each modality provides unique and complementary information suited to different application scenarios. Consequently, numerous studies have investigated diverse approaches for HAR using these modalities. This paper presents a comprehensive survey of the latest advancements in HAR from 2014 to 2024, focusing on machine learning (ML) and deep learning (DL) approaches categorized by input data modalities. We review both single-modality and multi-modality techniques, highlighting fusion-based and co-learning frameworks. Additionally, we cover advancements in hand-crafted action features, methods for recognizing human-object interactions, and activity detection. Our survey includes a detailed dataset description for each modality and a summary of the latest HAR systems, offering comparative results on benchmark datasets. Finally, we provide insightful observations and propose effective future research directions in HAR.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 42 of its 370 references that are in the map. First posted on arXiv 15 Sep 2024.

Builds on

42 of 370 references.

References · 42

  1. 2017Attention Is All You Needcited by 59,932
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  3. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  4. 2017The Kinetics Human Action Video Datasetcited by 1,894
  5. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,067
  6. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 789
  7. 2017Learning Spatio-Temporal Representation with Pseudo-3D Residual Networkscited by 386
  8. 2017Large-Scale Evolution of Image Classifierscited by 348
  9. 2018A Short Note about Kinetics-600cited by 331
  10. 2024InternVideo2: Scaling Foundation Models for Multimodal Video Understandingcited by 310
  11. 2019A Short Note on the Kinetics-700 Human Action Datasetcited by 308
  12. 2015Beyond Short Snippets: Deep Networks for Video Classificationcited by 233
30 more
  1. 2018Human Action Recognition and Prediction: A Surveycited by 226
  2. 2020Audiovisual SlowFast Networks for Video Recognitioncited by 144
  3. 2020Deep Learning for Sensor-based Human Activity Recognition: Overview, Challenges and Opportunitiescited by 125
  4. 2017Dataset Augmentation in Feature Spacecited by 124
  5. 2022UniFormerV2: Spatiotemporal Learning by Arming Image ViTs with Video UniFormercited by 116
  6. 2018GAN Augmentation: Augmenting Training Data using Generative Adversarial Networkscited by 107
  7. 2017PKU-MMD: A Large Scale Benchmark for Continuous Multi-Modal Human Action Understandingcited by 101
  8. 2020A Comprehensive Study of Deep Video Action Recognitioncited by 87
  9. 2018Pose Flow: Efficient Online Pose Trackingcited by 80
  10. 2020SMART Frame Selection for Action Recognitioncited by 73
  11. 2015Action Recognition using Visual Attentioncited by 49
  12. 2020V4D:4D Convolutional Neural Networks for Video-level Representation Learningcited by 33
  13. 2017Temporal 3D ConvNets: New Architecture and Transfer Learning for Video Classificationcited by 27
  14. 2023Language Knowledge-Assisted Representation Learning for Skeleton-Based Action Recognitioncited by 27
  15. 2014Articulated Pose Estimation by a Graphical Model with Image Dependent Pairwise Relationscited by 24
  16. 2017Two-Stream 3D Convolutional Neural Network for Skeleton-Based Action Recognitioncited by 23
  17. 2017PatchShuffle Regularizationcited by 23
  18. 2020RareAct: A video dataset of unusual interactionscited by 23
  19. 2018Part-based Graph Convolutional Network for Action Recognitioncited by 19
  20. 2018DiscrimNet: Semi-Supervised Action Recognition from Videos using Generative Adversarial Networkscited by 14
  21. 2018Learning More Robust Features with Adversarial Trainingcited by 8
  22. 2024Multi-modality action recognition based on dual feature shift in vehicle cabin monitoringcited by 8
  23. 2024Sensor Data Augmentation from Skeleton Pose Sequences for Improving Human Activity Recognitioncited by 8
  24. 2020Human in Events: A Large-Scale Benchmark for Human-centric Video Analysis in Complex Eventscited by 7
  25. 2020Multi Scale Temporal Graph Networks For Skeleton-based Action Recognitioncited by 5
  26. 2019Training Kinetics in 15 Minutes: Large-scale Distributed Training on Videoscited by 3
  27. 2024Deep Generative Domain Adaptation with Temporal Relation Knowledge for Cross-User Activity Recognitioncited by 3
  28. 2021Video Action Recognition Using spatio-temporal optical flow video framescited by 2
  29. 2023ConViViT -- A Deep Neural Network Combining Convolutions and Factorized Self-Attention for Human Activity Recognitioncited by 2
  30. 2024Self-supervised New Activity Detection in Sensor-based Smart Environmentscited by 1

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 20253D Skeleton-Based Action Recognition: A Reviewcited by 6
  2. 2025Skeleton-Based Transformer for Classification of Errors and Better Feedback in Low Back Pain Physical Rehabilitation Exercisescited by 1