# EgoEvGesture: Gesture Recognition Based on Egocentric Event Camera arXiv: https://arxiv.org/abs/2503.12419 ยท HTML: https://mlsanity.com/paper/2503.12419/ ## Key facts - Cited by 4 papers in the map (data through 14 Oct 2026). - Builds on 34 of its 63 references that are in the map. - First posted on arXiv 16 Mar 2025. ## Builds on (most cited first, 30 of 34) - [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019) - [HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interaction](https://mlsanity.com/paper/2203.01577/) (2022) - [ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation](https://mlsanity.com/paper/2204.13662/) (2022) - [Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activities](https://mlsanity.com/paper/2203.14712/) (2022) - [H2O: Two Hands Manipulating Objects for First Person Interaction Recognition](https://mlsanity.com/paper/2104.11181/) (2021) - [First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotations](https://mlsanity.com/paper/1704.02463/) (2017) - [The EPIC-KITCHENS Dataset: Collection, Challenges and Baselines](https://mlsanity.com/paper/2005.00343/) (2020) - [Keypoint Transformer: Solving Joint Identification in Challenging Hands and Object Interactions for Accurate 3D Pose Estimation](https://mlsanity.com/paper/2104.14639/) (2021) - [Real-time Hand Tracking under Occlusion from an Egocentric RGB-D Sensor](https://mlsanity.com/paper/1704.02201/) (2017) - [An Outlook into the Future of Egocentric Vision](https://mlsanity.com/paper/2308.07123/) (2023) - [BlazeFace: Sub-millisecond Neural Face Detection on Mobile GPUs](https://mlsanity.com/paper/1907.05047/) (2019) - [AssemblyHands: Towards Egocentric Activity Understanding via 3D Hand Pose Estimation](https://mlsanity.com/paper/2304.12301/) (2023) - [Event Based, Near Eye Gaze Tracking Beyond 10,000Hz](https://mlsanity.com/paper/2004.03577/) (2020) - [E2(GO)MOTION: Motion Augmented Event Stream for Egocentric Action Recognition](https://mlsanity.com/paper/2112.03596/) (2021) - [EventHands: Real-Time Neural 3D Hand Pose Estimation from an Event Stream](https://mlsanity.com/paper/2012.06475/) (2020) - [EventHPE: Event-based 3D Human Pose and Shape Estimation](https://mlsanity.com/paper/2108.06819/) (2021) - [Hierarchical Temporal Transformer for 3D Hand Pose Estimation and Action Recognition from Egocentric RGB Videos](https://mlsanity.com/paper/2209.09484/) (2022) - [Benchmarks and Challenges in Pose Estimation for Egocentric Hand Interactions with Objects](https://mlsanity.com/paper/2403.16428/) (2024) - [3ET: Efficient Event-based Eye Tracking using a Change-Based ConvLSTM Network](https://mlsanity.com/paper/2308.11771/) (2023) - [EventEgo3D: 3D Human Motion Capture from Egocentric Event Streams](https://mlsanity.com/paper/2404.08640/) (2024) - [Analysis of the hands in egocentric vision: A survey](https://mlsanity.com/paper/1912.10867/) (2019) - [Efficient Human Pose Estimation via 3D Event Point Cloud](https://mlsanity.com/paper/2206.04511/) (2022) - [EvHandPose: Event-based 3D Hand Pose Estimation with Sparse Supervision](https://mlsanity.com/paper/2303.02862/) (2023) - [3D Pose Estimation of Two Interacting Hands from a Monocular Event Camera](https://mlsanity.com/paper/2312.14157/) (2023) - [On the Utility of 3D Hand Poses for Action Recognition](https://mlsanity.com/paper/2403.09805/) (2024) - [CNN+RNN Depth and Skeleton based Dynamic Hand Gesture Recognition](https://mlsanity.com/paper/2007.11983/) (2020) - [Complementing Event Streams and RGB Frames for Hand Mesh Reconstruction](https://mlsanity.com/paper/2403.07346/) (2024) - [Event-based Gesture Recognition with Dynamic Background Suppression using Smartphone Computational Capabilities](https://mlsanity.com/paper/1811.07802/) (2018) - [In My Perspective, In My Hands: Accurate Egocentric 2D Hand Pose and Action Recognition](https://mlsanity.com/paper/2404.09308/) (2024) - [Helios: An extremely low power event-based gesture recognition for always-on smart eyewear](https://mlsanity.com/paper/2407.05206/) (2024) ## Built on by (most cited first, 4 of 4) - [EgoEV-HandPose: Egocentric 3D Hand Pose Estimation and Gesture Recognition with Stereo Event Cameras](https://mlsanity.com/paper/2605.12297/) (2026) - [OpenGlass: Ultra-Low-Power On-Device AI Eyewear with Event-based Vision](https://mlsanity.com/paper/2606.07431/) (2026) - [NeuroLip: An Event-driven Spatiotemporal Learning Framework for Cross-Scene Lip-Motion-based Visual Speaker Recognition](https://mlsanity.com/paper/2604.15718/) (2026) - [Fully-sensorized smart-eyewear platform for on-device Machine Learning](https://mlsanity.com/paper/2607.16222/) (2026) ## Abstract (arXiv) Egocentric gesture recognition is a pivotal technology for enhancing natural human-computer interaction, yet traditional RGB-based solutions suffer from motion blur and illumination variations in dynamic scenarios. While event cameras show distinct advantages in handling high dynamic range with ultra-low power consumption, existing RGB-based architectures face inherent limitations in processing asynchronous event streams due to their synchronous frame-based nature. Moreover, from an egocentric perspective, event cameras record data that includes events generated by both head movements and hand gestures, thereby increasing the complexity of gesture recognition. To address this, we propose a novel network architecture specifically designed for event data processing, incorporating (1) a lightweight CNN with asymmetric depthwise convolutions to reduce parameters while preserving spatiotemporal features, (2) a plug-and-play state-space model as context block that decouples head movement noise from gesture dynamics, and (3) a parameter-free Bins-Temporal Shift Module (BTSM) that shifts features along bins and temporal dimensions to fuse sparse events efficiently. We further establish the EgoEvGesture dataset, the first large-scale dataset for egocentric gesture recognition using event cameras. Experimental results demonstrate that our method achieves 62.7% accuracy tested on unseen subjects with only 7M parameters, 3.1% higher than state-of-the-art approaches. Notable misclassifications in freestyle motions stem from high inter-personal variability and unseen test patterns differing from training data. Moreover, our approach achieved a remarkable accuracy of 97.0% on the DVS128 Gesture, demonstrating the effectiveness and generalization capability of our method on public datasets. The dataset and models are made available at https://github.com/3190105222/EgoEv_Gesture.