# An Advanced Deep Learning Based Three-Stream Hybrid Model for Dynamic Hand Gesture Recognition

arXiv: https://arxiv.org/abs/2408.08035 · HTML: https://mlsanity.com/paper/2408.08035/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 43 references that are in the map.
- First posted on arXiv 15 Aug 2024.

## Builds on (most cited first, 2 of 2)

- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [A Methodological and Structural Review of Hand Gesture Recognition Across Diverse Data Modalities](https://mlsanity.com/paper/2408.05436/) (2024)

## Built on by (most cited first, 4 of 4)

- [Multimodal Attention-Enhanced Feature Fusion-based Weekly Supervised Anomaly Violence Detection](https://mlsanity.com/paper/2409.11223/) (2024)
- [Bengali Sign Language Recognition through Hand Pose Estimation using Multi-Branch Spatial-Temporal Attention Model](https://mlsanity.com/paper/2408.14111/) (2024)
- [FireLite: Leveraging Transfer Learning for Efficient Fire Detection in Resource-Constrained Environments](https://mlsanity.com/paper/2409.20384/) (2024)
- [Duo Streamers: A Streaming Gesture Recognition Framework](https://mlsanity.com/paper/2502.12297/) (2025)

## Abstract (arXiv)

In the modern context, hand gesture recognition has emerged as a focal point. This is due to its wide range of applications, which include comprehending sign language, factories, hands-free devices, and guiding robots. Many researchers have attempted to develop more effective techniques for recognizing these hand gestures. However, there are challenges like dataset limitations, variations in hand forms, external environments, and inconsistent lighting conditions. To address these challenges, we proposed a novel three-stream hybrid model that combines RGB pixel and skeleton-based features to recognize hand gestures. In the procedure, we preprocessed the dataset, including augmentation, to make rotation, translation, and scaling independent systems. We employed a three-stream hybrid model to extract the multi-feature fusion using the power of the deep learning module. In the first stream, we extracted the initial feature using the pre-trained Imagenet module and then enhanced this feature by using a multi-layer of the GRU and LSTM modules. In the second stream, we extracted the initial feature with the pre-trained ReseNet module and enhanced it with the various combinations of the GRU and LSTM modules. In the third stream, we extracted the hand pose key points using the media pipe and then enhanced them using the stacked LSTM to produce the hierarchical feature. After that, we concatenated the three features to produce the final. Finally, we employed a classification module to produce the probabilistic map to generate predicted output. We mainly produced a powerful feature vector by taking advantage of the pixel-based deep learning feature and pos-estimation-based stacked deep learning feature, including a pre-trained model with a scratched deep learning model for unequalled gesture detection capabilities.