# Motion Fused Frames: Data Level Fusion Strategy for Hand Gesture Recognition

arXiv: https://arxiv.org/abs/1804.07187 · HTML: https://mlsanity.com/paper/1804.07187/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Apr 2018.

## Built on by (most cited first, 15 of 15)

- [How to Design a Three-Stage Architecture for Audio-Visual Active Speaker Detection in the Wild](https://mlsanity.com/paper/2106.03932/) (2021)
- [Combined Person Classification with Airborne Optical Sectioning](https://mlsanity.com/paper/2106.10077/) (2021)
- [Survey on Hand Gesture Recognition from Visual Input](https://mlsanity.com/paper/2501.11992/) (2025)
- [M3L: Language-based Video Editing via Multi-Modal Multi-Level Transformers](https://mlsanity.com/paper/2104.01122/) (2021)
- [A Methodological and Structural Review of Hand Gesture Recognition Across Diverse Data Modalities](https://mlsanity.com/paper/2408.05436/) (2024)
- [GestFormer: Multiscale Wavelet Pooling Transformer Network for Dynamic Hand Gesture Recognition](https://mlsanity.com/paper/2405.11180/) (2024)
- [A Deep Learning-based Multimodal Depth-Aware Dynamic Hand Gesture Recognition System](https://mlsanity.com/paper/2107.02543/) (2021)
- [Real-Time Hand Gesture Recognition: Integrating Skeleton-Based Data Fusion and Multi-Stream CNN](https://mlsanity.com/paper/2406.15003/) (2024)
- [MVTN: A Multiscale Video Transformer Network for Hand Gesture Recognition](https://mlsanity.com/paper/2409.03890/) (2024)
- [ConvMixFormer- A Resource-efficient Convolution Mixer for Transformer-based Dynamic Hand Gesture Recognition](https://mlsanity.com/paper/2411.07118/) (2024)
- [Multiscaled Multi-Head Attention-based Video Transformer Network for Hand Gesture Recognition](https://mlsanity.com/paper/2501.00935/) (2025)
- [Multi-task Learning For Joint Action and Gesture Recognition](https://mlsanity.com/paper/2505.17867/) (2025)
- [SegSLR: Promptable Video Segmentation for Isolated Sign Language Recognition](https://mlsanity.com/paper/2509.10710/) (2025)
- [Online Hand Gesture Recognition Using 3D Convolutional Neural Networks](https://mlsanity.com/paper/2605.23409/) (2026)
- [UniCon-Former: Unified Convolution Transformer is All You Need for Hand Gesture Recognition](https://mlsanity.com/paper/2608.13217/) (2026)

## Abstract (arXiv)

Acquiring spatio-temporal states of an action is the most crucial step for action classification. In this paper, we propose a data level fusion strategy, Motion Fused Frames (MFFs), designed to fuse motion information into static images as better representatives of spatio-temporal states of an action. MFFs can be used as input to any deep learning architecture with very little modification on the network. We evaluate MFFs on hand gesture recognition tasks using three video datasets - Jester, ChaLearn LAP IsoGD and NVIDIA Dynamic Hand Gesture Datasets - which require capturing long-term temporal relations of hand movements. Our approach obtains very competitive performance on Jester and ChaLearn benchmarks with the classification accuracies of 96.28% and 57.4%, respectively, while achieving state-of-the-art performance with 84.7% accuracy on NVIDIA benchmark.