mlsanityData to 7 Oct 2026

Human Action Recognition from Various Data Modalities: A Review

arXiv 2012.11866 ↗First posted 22 Dec 2020cs.CVcited by 162

Not on the map. Why?

Abstract (arXiv)

Human Action Recognition (HAR) aims to understand human behavior and assign a label to each action. It has a wide range of applications, and therefore has been attracting increasing attention in the field of computer vision. Human actions can be represented using various data modalities, such as RGB, skeleton, depth, infrared, point cloud, event stream, audio, acceleration, radar, and WiFi signal, which encode different sources of useful yet distinct information and have various advantages depending on the application scenarios. Consequently, lots of existing works have attempted to investigate different types of approaches for HAR using various modalities. In this paper, we present a comprehensive survey of recent progress in deep learning methods for HAR based on the type of input data modality. Specifically, we review the current mainstream deep learning methods for single data modalities and multiple data modalities, including the fusion-based and the co-learning-based frameworks. We also present comparative results on several benchmark datasets for HAR, together with insightful observations and inspiring future research directions.

Key facts. Cited by 162 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 20; 2023: 28; 2024: 52; 2025: 39; 2026: 17.

162 papers.

Sort:

By area:

  1. 2022CLIP-TSA: CLIP-Assisted Temporal Self-Attention for Weakly-Supervised Video Anomaly Detectioncited by 67
  2. 2021Efficient Video Transformers with Spatial-Temporal Token Selectioncited by 58
  3. 2025UAVs Meet LLMs: Overviews and Perspectives Toward Agentic Low-Altitude Mobilitycited by 48
  4. 2024SkateFormer: Skeletal-Temporal Transformer for Human Action Recognitioncited by 46
  5. 2022Learning in Audio-visual Context: A Review, Analysis, and New Perspectivecited by 44
  6. 2023M3Net: Multi-view Encoding, Matching, and Fusion for Few-shot Fine-grained Action Recognitioncited by 34
  7. 2024ExACT: Language-guided Conceptual Reasoning and Uncertainty Estimation for Event-based Action Recognition and Morecited by 33
  8. 2022SkeletonMAE: Spatial-Temporal Masked Autoencoders for Self-supervised Skeleton Action Recognitioncited by 29
  9. 2022A Survey on Video Action Recognition in Sports: Datasets, Methods and Applicationscited by 27
  10. 2023Language Knowledge-Assisted Representation Learning for Skeleton-Based Action Recognitioncited by 27
  11. 2024Deep Learning for Video Anomaly Detection: A Reviewcited by 27
  12. 2021Cross-modal Knowledge Distillation for Vision-to-Sensor Action Recognitioncited by 25
38 more
  1. 2022Vision Transformers for Action Recognition: A Surveycited by 25
  2. 2022Towards a Unified View on Visual Parameter-Efficient Transfer Learningcited by 25
  3. 2024Part-aware Unified Representation of Language and Skeleton for Zero-shot Action Recognitioncited by 23
  4. 2024Networking Systems for Video Anomaly Detection: A Tutorial and Surveycited by 21
  5. 2023MDCS: More Diverse Experts with Consistency Self-distillation for Long-tailed Recognitioncited by 19
  6. 2023Diversifying Spatial-Temporal Perception for Video Domain Generalizationcited by 19
  7. 2023DVANet: Disentangling View and Action Features for Multi-View Action Recognitioncited by 19
  8. 2022A Coding Framework and Benchmark towards Low-Bitrate Video Understandingcited by 17
  9. 2022Contrastive Learning from Spatio-Temporal Mixed Skeleton Sequences for Self-Supervised Skeleton-Based Action Recognitioncited by 17
  10. 2024RT-Pose: A 4D Radar Tensor-based 3D Human Pose Estimation and Localization Benchmarkcited by 16
  11. 2024XOV-Action: Towards Generalizable Open-Vocabulary Action Recognitioncited by 15
  12. 2024Learning to Rebalance Multi-Modal Optimization by Adaptively Masking Subnetworkscited by 15
  13. 2023Computation-efficient Deep Learning for Computer Vision: A Surveycited by 14
  14. 2023A Survey on Deep Learning Techniques for Action Anticipationcited by 11
  15. 2024CrossGLG: LLM Guides One-shot Skeleton-based 3D Action Recognition in a Cross-level Mannercited by 11
  16. 2024A Comprehensive Review of Few-shot Action Recognitioncited by 11
  17. 2024Event Stream based Human Action Recognition: A High-Definition Benchmark Dataset and Algorithmscited by 11
  18. 2024Privacy-Preserving Video Anomaly Detection: A Surveycited by 11
  19. 2025Temporal-Guided Spiking Neural Networks for Event-Based Human Action Recognitioncited by 11
  20. 2021Adversarial Bone Length Attack on Action Recognitioncited by 10
  21. 2023EventTransAct: A video transformer-based framework for Event-camera based action recognitioncited by 10
  22. 2023CAD -- Contextual Multi-modal Alignment for Dynamic AVQAcited by 10
  23. 2024A Survey on Multimodal Wearable Sensor-based Human Action Recognitioncited by 10
  24. 2024NarrativeBridge: Enhancing Video Captioning with Causal-Temporal Narrativecited by 10
  25. 2021TedNet: A Pytorch Toolkit for Tensor Decomposition Networkscited by 9
  26. 2022Contrastive Learning with Cross-Modal Knowledge Mining for Multimodal Human Activity Recognitioncited by 9
  27. 2022Adaptive Local-Component-aware Graph Convolutional Network for One-shot Skeleton-based Action Recognitioncited by 9
  28. 2023D-STGCNT: A Dense Spatio-Temporal Graph Conv-GRU Network based on transformer for assessment of patient physical rehabilitationcited by 9
  29. 2024A Backpack Full of Skills: Egocentric Video Understanding with Diverse Task Perspectivescited by 9
  30. 2024Bridging the Skeleton-Text Modality Gap: Diffusion-Powered Modality Alignment for Zero-shot Skeleton-based Action Recognitioncited by 9
  31. 2024Language Model Guided Interpretable Video Action Reasoningcited by 8
  32. 2024Human-Centric Transformer for Domain Adaptive Action Recognitioncited by 8
  33. 2024Human Action Anticipation: A Surveycited by 8
  34. 2024HumanVLM: Foundation for Human-Scene Vision-Language Modelcited by 8
  35. 2022End-to-end Transformer for Compressed Video Quality Enhancementcited by 7
  36. 2023Generative Model-based Feature Knowledge Distillation for Action Recognitioncited by 7
  37. 2024From CNNs to Transformers in Multimodal Human Action Recognition: A Surveycited by 7
  38. 2024TASAR: Transfer-based Attack on Skeletal Action Recognitioncited by 7

50 most cited of 162 All 162 (JSON)