mlsanityData to 13 Oct 2026

MVTN: A Multiscale Video Transformer Network for Hand Gesture Recognition

arXiv 2409.03890 ↗First posted 5 Sep 2024cs.CV cs.HCcited by 2

Not on the map. Why?

Abstract (arXiv)

In this paper, we introduce a novel Multiscale Video Transformer Network (MVTN) for dynamic hand gesture recognition, since multiscale features can extract features with variable size, pose, and shape of hand which is a challenge in hand gesture recognition. The proposed model incorporates a multiscale feature hierarchy to capture diverse levels of detail and context within hand gestures which enhances the model's ability. This multiscale hierarchy is obtained by extracting different dimensions of attention in different transformer stages with initial stages to model high-resolution features and later stages to model low-resolution features. Our approach also leverages multimodal data, utilizing depth maps, infrared data, and surface normals along with RGB images from NVGesture and Briareo datasets. Experiments show that the proposed MVTN achieves state-of-the-art results with less computational complexity and parameters. The source code is available at https://github.com/mallikagarg/MVTN.

Key facts. Cited by 2 papers in the map (data through 13 Oct 2026). Builds on 38 of its 57 references that are in the map. First posted on arXiv 5 Sep 2024.

Builds on

38 of 57 references.

References · 38

  1. 2017Attention Is All You Needcited by 60,106
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,950
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,043
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,111
  5. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,264
  6. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,961
  7. 2020Rethinking Attention with Performerscited by 1,594
  8. 2020Linformer: Self-Attention with Linear Complexitycited by 1,469
  9. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,067
  10. 2021CvT: Introducing Convolutions to Vision Transformerscited by 946
  11. 2021Multiscale Vision Transformerscited by 896
  12. 2014Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognitioncited by 720
26 more
  1. 2021Twins: Revisiting the Design of Spatial Attention in Vision Transformerscited by 565
  2. 2021CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windowscited by 527
  3. 2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 517
  4. 2021LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inferencecited by 475
  5. 2021Nystr"omformer: A Nystr"om-Based Algorithm for Approximating Self-Attentioncited by 469
  6. 2021Early Convolutions Help Transformers See Bettercited by 396
  7. 2021Rethinking Spatial Dimensions of Vision Transformerscited by 361
  8. 2021Video Transformer Networkcited by 271
  9. 2021Incorporating Convolution Designs into Visual Transformerscited by 248
  10. 2022UniFormer: Unifying Convolution and Self-attention for Visual Recognitioncited by 206
  11. 2021HRFormer: High-Resolution Transformer for Dense Predictioncited by 149
  12. 2022CenterFormer: Center-based Transformer for 3D Object Detectioncited by 99
  13. 2021Multi-Scale High-Resolution Vision Transformer for Semantic Segmentationcited by 92
  14. 2021P2T: Pyramid Pooling Transformer for Scene Understandingcited by 84
  15. 2020Searching Multi-Rate and Multi-Modal Temporal Enhanced Networks for Gesture Recognitioncited by 36
  16. 2023POTTER: Pooling Attention Transformer for Efficient Human Mesh Recoverycited by 31
  17. 2019Real-time Hand Gesture Detection and Classification Using Convolutional Neural Networkscited by 30
  18. 2018Improving the Performance of Unimodal Dynamic Hand-Gesture Recognition with Multimodal Trainingcited by 28
  19. 2021PSViT: Better Vision Transformer via Token Pooling and Attention Sharingcited by 21
  20. 2015A robust and efficient video representation for action recognitioncited by 20
  21. 2018Motion Fused Frames: Data Level Fusion Strategy for Hand Gesture Recognitioncited by 15
  22. 2020Res3ATN -- Deep 3D Residual Attention Network for Hand Gesture Recognition in Videoscited by 10
  23. 2022Attention guided global enhancement and local refinement network for semantic segmentationcited by 10
  24. 2022Efficacy of Transformer Networks for Classification of Raw EEG Datacited by 9
  25. 2021Multi-Task and Multi-Modal Learning for RGB Dynamic Gesture Recognitioncited by 7
  26. 2024GestFormer: Multiscale Wavelet Pooling Transformer Network for Dynamic Hand Gesture Recognitioncited by 7

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2024: 1; 2025: 0; 2026: 1.

2 papers.

  1. 2024ConvMixFormer- A Resource-efficient Convolution Mixer for Transformer-based Dynamic Hand Gesture Recognitioncited by 2
  2. 2026UniCon-Former: Unified Convolution Transformer is All You Need for Hand Gesture Recognitioncited by 0