# An Ensemble of Knowledge Sharing Models for Dynamic Hand Gesture Recognition

arXiv: https://arxiv.org/abs/2008.05732 · HTML: https://mlsanity.com/paper/2008.05732/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Aug 2020.

## Built on by (most cited first, 4 of 4)

- [A Deep Learning-based Multimodal Depth-Aware Dynamic Hand Gesture Recognition System](https://mlsanity.com/paper/2107.02543/) (2021)
- [Real-Time Hand Gesture Recognition: Integrating Skeleton-Based Data Fusion and Multi-Stream CNN](https://mlsanity.com/paper/2406.15003/) (2024)
- [Fine-grained Hand Gesture Recognition in Multi-viewpoint Hand Hygiene](https://mlsanity.com/paper/2109.02917/) (2021)
- [Hand Gesture Classification on Praxis Dataset: Trading Accuracy for Expense](https://mlsanity.com/paper/2311.00767/) (2023)

## Abstract (arXiv)

The focus of this paper is dynamic gesture recognition in the context of the interaction between humans and machines. We propose a model consisting of two sub-networks, a transformer and an ordered-neuron long-short-term-memory (ON-LSTM) based recurrent neural network (RNN). Each sub-network is trained to perform the task of gesture recognition using only skeleton joints. Since each sub-network extracts different types of features due to the difference in architecture, the knowledge can be shared between the sub-networks. Through knowledge distillation, the features and predictions from each sub-network are fused together into a new fusion classifier. In addition, a cyclical learning rate can be used to generate a series of models that are combined in an ensemble, in order to yield a more generalizable prediction. The proposed ensemble of knowledge-sharing models exhibits an overall accuracy of 86.11% using only skeleton information, as tested using the Dynamic Hand Gesture-14/28 dataset