# RGB-D-based Human Motion Recognition with Deep Learning: A Survey

arXiv: https://arxiv.org/abs/1711.08362 · HTML: https://mlsanity.com/paper/1711.08362/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2017.

## Built on by (most cited first, 30 of 37)

- [Temporal Attention Unit: Towards Efficient Spatiotemporal Predictive Learning](https://mlsanity.com/paper/2206.12126/) (2022)
- [Multi-Scale Spatial Temporal Graph Convolutional Network for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2206.13028/) (2022)
- [OpenSTL: A Comprehensive Benchmark of Spatio-Temporal Predictive Learning](https://mlsanity.com/paper/2306.11249/) (2023)
- [Trear: Transformer-based RGB-D Egocentric Action Recognition](https://mlsanity.com/paper/2101.03904/) (2021)
- [VMRNN: Integrating Vision Mamba and LSTM for Efficient and Accurate Spatiotemporal Forecasting](https://mlsanity.com/paper/2403.16536/) (2024)
- [Augmented Transformer with Adaptive Graph for Temporal Action Proposal Generation](https://mlsanity.com/paper/2103.16024/) (2021)
- [Focal and Global Spatial-Temporal Transformer for Skeleton-based Action Recognition](https://mlsanity.com/paper/2210.02693/) (2022)
- [InsPose: Instance-Aware Networks for Single-Stage Multi-Person Pose Estimation](https://mlsanity.com/paper/2107.08982/) (2021)
- [TIMo -- A Dataset for Indoor Building Monitoring with a Time-of-Flight Camera](https://mlsanity.com/paper/2108.12196/) (2021)
- [Triplet Attention Transformer for Spatiotemporal Predictive Learning](https://mlsanity.com/paper/2310.18698/) (2023)
- [From CNNs to Transformers in Multimodal Human Action Recognition: A Survey](https://mlsanity.com/paper/2405.15813/) (2024)
- [A Central Difference Graph Convolutional Operator for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2111.06995/) (2021)
- [Human Pose-based Estimation, Tracking and Action Recognition with Deep Learning: A Survey](https://mlsanity.com/paper/2310.13039/) (2023)
- [Video Prediction Transformers without Recurrence or Convolution](https://mlsanity.com/paper/2410.04733/) (2024)
- [Human Body Pose Estimation for Gait Identification: A Comprehensive Survey of Datasets and Models](https://mlsanity.com/paper/2305.13765/) (2023)
- [Single Person Pose Estimation: A Survey](https://mlsanity.com/paper/2109.10056/) (2021)
- [Learning from Temporal Spatial Cubism for Cross-Dataset Skeleton-based Action Recognition](https://mlsanity.com/paper/2207.08095/) (2022)
- [Spatio-Temporal Joint Density Driven Learning for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2505.23012/) (2025)
- [Transformers in Action Recognition: A Review on Temporal Modeling](https://mlsanity.com/paper/2302.01921/) (2022)
- [Joint Temporal Pooling for Improving Skeleton-based Action Recognition](https://mlsanity.com/paper/2408.09356/) (2024)
- [Multi-Scale Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition](https://mlsanity.com/paper/2111.03993/) (2021)
- [Multi-stage Factorized Spatio-Temporal Representation for RGB-D Action and Gesture Recognition](https://mlsanity.com/paper/2308.12006/) (2023)
- [A vision-based framework for human behavior understanding in industrial assembly lines](https://mlsanity.com/paper/2409.17356/) (2024)
- [Adversarial Interaction Attack: Fooling AI to Misinterpret Human Intentions](https://mlsanity.com/paper/2101.06704/) (2021)
- [Bridging the gap between Human Action Recognition and Online Action Detection](https://mlsanity.com/paper/2101.08851/) (2021)
- [Learning Chebyshev Basis in Graph Convolutional Networks for Skeleton-based Action Recognition](https://mlsanity.com/paper/2104.05482/) (2021)
- [Two-stream Multi-level Dynamic Point Transformer for Two-person Interaction Recognition](https://mlsanity.com/paper/2307.11973/) (2023)
- [Unveiling the frontiers of deep learning: innovations shaping diverse domains](https://mlsanity.com/paper/2309.02712/) (2023)
- [One-Shot Multi-Rate Pruning of Graph Convolutional Networks](https://mlsanity.com/paper/2312.17615/) (2023)
- [Active Generation Network of Human Skeleton for Action Recognition](https://mlsanity.com/paper/2401.17086/) (2024)

## Abstract (arXiv)

Human motion recognition is one of the most important branches of human-centered research activities. In recent years, motion recognition based on RGB-D data has attracted much attention. Along with the development in artificial intelligence, deep learning techniques have gained remarkable success in computer vision. In particular, convolutional neural networks (CNN) have achieved great success for image-based tasks, and recurrent neural networks (RNN) are renowned for sequence-based problems. Specifically, deep learning methods based on the CNN and RNN architectures have been adopted for motion recognition using RGB-D data. In this paper, a detailed overview of recent advances in RGB-D-based motion recognition is presented. The reviewed methods are broadly categorized into four groups, depending on the modality adopted for recognition: RGB-based, depth-based, skeleton-based and RGB+D-based. As a survey focused on the application of deep learning to RGB-D-based motion recognition, we explicitly discuss the advantages and limitations of existing techniques. Particularly, we highlighted the methods of encoding spatial-temporal-structural information inherent in video sequence, and discuss potential directions for future research.