# CNN+RNN Depth and Skeleton based Dynamic Hand Gesture Recognition

arXiv: https://arxiv.org/abs/2007.11983 · HTML: https://mlsanity.com/paper/2007.11983/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jul 2020.

## Built on by (most cited first, 16 of 16)

- [Survey on Hand Gesture Recognition from Visual Input](https://mlsanity.com/paper/2501.11992/) (2025)
- [The Imaginative Generative Adversarial Network: Automatic Data Augmentation for Dynamic Skeleton-Based Hand Gesture and Human Action Recognition](https://mlsanity.com/paper/2105.13061/) (2021)
- [A Methodological and Structural Review of Hand Gesture Recognition Across Diverse Data Modalities](https://mlsanity.com/paper/2408.05436/) (2024)
- [A Deep Learning-based Multimodal Depth-Aware Dynamic Hand Gesture Recognition System](https://mlsanity.com/paper/2107.02543/) (2021)
- [EgoEvGesture: Gesture Recognition Based on Egocentric Event Camera](https://mlsanity.com/paper/2503.12419/) (2025)
- [Transformers in Action Recognition: A Review on Temporal Modeling](https://mlsanity.com/paper/2302.01921/) (2022)
- [DSTSA-GCN: Advancing Skeleton-Based Gesture Recognition with Semantic-Aware Spatio-Temporal Topology Modeling](https://mlsanity.com/paper/2501.12086/) (2025)
- [Real-Time Hand Gesture Recognition: Integrating Skeleton-Based Data Fusion and Multi-Stream CNN](https://mlsanity.com/paper/2406.15003/) (2024)
- [Fine-grained Hand Gesture Recognition in Multi-viewpoint Hand Hygiene](https://mlsanity.com/paper/2109.02917/) (2021)
- [Online hand gesture recognition using Continual Graph Transformers](https://mlsanity.com/paper/2502.14939/) (2025)
- [UAC: Uncertainty-Aware Calibration of Neural Networks for Gesture Detection](https://mlsanity.com/paper/2504.02895/) (2025)
- [Dynamic Gesture Recognition](https://mlsanity.com/paper/2109.09396/) (2021)
- [Hand Me Your PIN! Inferring ATM PINs of Users Typing with a Covered Hand](https://mlsanity.com/paper/2110.08113/) (2021)
- [Hand Gesture Classification on Praxis Dataset: Trading Accuracy for Expense](https://mlsanity.com/paper/2311.00767/) (2023)
- [Study and Survey on Gesture Recognition Systems](https://mlsanity.com/paper/2312.00392/) (2023)
- [Wearable environmental sensing to forecast how legged systems will interact with upcoming terrain](https://mlsanity.com/paper/2602.09209/) (2026)

## Abstract (arXiv)

Human activity and gesture recognition is an important component of rapidly growing domain of ambient intelligence, in particular in assisting living and smart homes. In this paper, we propose to combine the power of two deep learning techniques, the convolutional neural networks (CNN) and the recurrent neural networks (RNN), for automated hand gesture recognition using both depth and skeleton data. Each of these types of data can be used separately to train neural networks to recognize hand gestures. While RNN were reported previously to perform well in recognition of sequences of movement for each skeleton joint given the skeleton information only, this study aims at utilizing depth data and apply CNN to extract important spatial information from the depth images. Together, the tandem CNN+RNN is capable of recognizing a sequence of gestures more accurately. As well, various types of fusion are studied to combine both the skeleton and depth information in order to extract temporal-spatial information. An overall accuracy of 85.46% is achieved on the dynamic hand gesture-14/28 dataset.