# EHWGesture -- A dataset for multimodal understanding of clinical gestures

arXiv: https://arxiv.org/abs/2509.07525 · HTML: https://mlsanity.com/paper/2509.07525/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 35 references that are in the map.
- First posted on arXiv 9 Sep 2025.

## Builds on (most cited first, 16 of 16)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotations](https://mlsanity.com/paper/1704.02463/) (2017)
- [MediaPipe Hands: On-device Real-time Hand Tracking](https://mlsanity.com/paper/2006.10214/) (2020)
- [Graph-Based Object Classification for Neuromorphic Vision Sensing](https://mlsanity.com/paper/1908.06648/) (2019)
- [Resource Efficient 3D Convolutional Neural Networks](https://mlsanity.com/paper/1904.02422/) (2019)
- [HaGRID - HAnd Gesture Recognition Image Dataset](https://mlsanity.com/paper/2206.08219/) (2022)
- [IPN Hand: A Video Dataset and Benchmark for Real-Time Continuous Hand Gesture Recognition](https://mlsanity.com/paper/2005.02134/) (2020)
- [PECoP: Parameter Efficient Continual Pretraining for Action Quality Assessment](https://mlsanity.com/paper/2311.07603/) (2023)
- [PhiNets: a scalable backbone for low-power AI at the edge](https://mlsanity.com/paper/2110.00337/) (2021)
- [Event-based Gesture Recognition with Dynamic Background Suppression using Smartphone Computational Capabilities](https://mlsanity.com/paper/1811.07802/) (2018)
- [Spatiotemporal Filtering for Event-Based Action Recognition](https://mlsanity.com/paper/1903.07067/) (2019)
- [Hand tracking for clinical applications: validation of the Google MediaPipe Hand (GMH) and the depth-enhanced GMH-D frameworks](https://mlsanity.com/paper/2308.01088/) (2023)
- [HaGRIDv2: 1M Images for Static and Dynamic Hand Gesture Recognition](https://mlsanity.com/paper/2412.01508/) (2024)

## Built on by (most cited first, 2 of 2)

- [EventBench: Towards Comprehensive Benchmarking of Event-based MLLMs](https://mlsanity.com/paper/2511.18448/) (2025)
- [Elastic Spiking Transformers for Efficient Gesture Understanding](https://mlsanity.com/paper/2605.13869/) (2026)

## Abstract (arXiv)

Hand gesture understanding is essential for several applications in human-computer interaction, including automatic clinical assessment of hand dexterity. While deep learning has advanced static gesture recognition, dynamic gesture understanding remains challenging due to complex spatiotemporal variations. Moreover, existing datasets often lack multimodal and multi-view diversity, precise ground-truth tracking, and an action quality component embedded within gestures. This paper introduces EHWGesture, a multimodal video dataset for gesture understanding featuring five clinically relevant gestures. It includes over 1,100 recordings (6 hours), captured from 25 healthy subjects using two high-resolution RGB-Depth cameras and an event camera. A motion capture system provides precise ground-truth hand landmark tracking, and all devices are spatially calibrated and synchronized to ensure cross-modal alignment. Moreover, to embed an action quality task within gesture understanding, collected recordings are organized in classes of execution speed that mirror clinical evaluations of hand dexterity. Baseline experiments highlight the dataset's potential for gesture classification, gesture trigger detection, and action quality assessment. Thus, EHWGesture can serve as a comprehensive benchmark for advancing multimodal clinical gesture understanding.