# Beyond Temporal Pooling: Recurrence and Temporal Convolutions for Gesture Recognition in Video

arXiv: https://arxiv.org/abs/1506.01911 · HTML: https://mlsanity.com/paper/1506.01911/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jun 2015.

## Built on by (most cited first, 15 of 15)

- [Skeleton Aware Multi-modal Sign Language Recognition](https://mlsanity.com/paper/2103.08833/) (2021)
- [Sign Language Recognition via Skeleton-Aware Multi-Model Ensemble](https://mlsanity.com/paper/2110.06161/) (2021)
- [Is context all you need? Scaling Neural Sign Language Translation to Large Domains of Discourse](https://mlsanity.com/paper/2308.09622/) (2023)
- [Using Motion History Images with 3D Convolutional Networks in Isolated Sign Language Recognition](https://mlsanity.com/paper/2110.12396/) (2021)
- [Survey on Hand Gesture Recognition from Visual Input](https://mlsanity.com/paper/2501.11992/) (2025)
- [StepNet: Spatial-temporal Part-aware Network for Isolated Sign Language Recognition](https://mlsanity.com/paper/2212.12857/) (2022)
- [Continual Inference: A Library for Efficient Online Inference with Deep Neural Networks in PyTorch](https://mlsanity.com/paper/2204.03418/) (2022)
- [A Methodological and Structural Review of Hand Gesture Recognition Across Diverse Data Modalities](https://mlsanity.com/paper/2408.05436/) (2024)
- [Hierarchical growing grid networks for skeleton based action recognition](https://mlsanity.com/paper/2104.11165/) (2021)
- [Siamese Networks for Weakly Supervised Human Activity Recognition](https://mlsanity.com/paper/2307.08944/) (2023)
- [Word-level Sign Language Recognition with Multi-stream Neural Networks Focusing on Local Regions and Skeletal Information](https://mlsanity.com/paper/2106.15989/) (2021)
- [Transformers in Action Recognition: A Review on Temporal Modeling](https://mlsanity.com/paper/2302.01921/) (2022)
- [Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions](https://mlsanity.com/paper/2108.13055/) (2021)
- [Comparative Analysis of XGBoost and Minirocket Algortihms for Human Activity Recognition](https://mlsanity.com/paper/2402.18296/) (2024)
- [Towards Diverse Perspective Learning with Selection over Multiple Temporal Poolings](https://mlsanity.com/paper/2403.09749/) (2024)

## Abstract (arXiv)

Recent studies have demonstrated the power of recurrent neural networks for machine translation, image captioning and speech recognition. For the task of capturing temporal structure in video, however, there still remain numerous open research questions. Current research suggests using a simple temporal feature pooling strategy to take into account the temporal aspect of video. We demonstrate that this method is not sufficient for gesture recognition, where temporal information is more discriminative compared to general video classification tasks. We explore deep architectures for gesture recognition in video and propose a new end-to-end trainable neural network architecture incorporating temporal convolutions and bidirectional recurrence. Our main contributions are twofold; first, we show that recurrence is crucial for this task; second, we show that adding temporal convolutions leads to significant improvements. We evaluate the different approaches on the Montalbano gesture recognition dataset, where we achieve state-of-the-art results.