mlsanityData to 7 Oct 2026

Helios: An extremely low power event-based gesture recognition for always-on smart eyewear

arXiv 2407.05206 ↗First posted 6 Jul 2024cs.CV cs.HC cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

This paper introduces Helios, the first extremely low-power, real-time, event-based hand gesture recognition system designed for all-day on smart eyewear. As augmented reality (AR) evolves, current smart glasses like the Meta Ray-Bans prioritize visual and wearable comfort at the expense of functionality. Existing human-machine interfaces (HMIs) in these devices, such as capacitive touch and voice controls, present limitations in ergonomics, privacy and power consumption. Helios addresses these challenges by leveraging natural hand interactions for a more intuitive and comfortable user experience. Our system utilizes a extremely low-power and compact 3mmx4mm/20mW event camera to perform natural hand-based gesture recognition for always-on smart eyewear. The camera's output is processed by a convolutional neural network (CNN) running on a NXP Nano UltraLite compute platform, consuming less than 350mW. Helios can recognize seven classes of gestures, including subtle microgestures like swipes and pinches, with 91% accuracy. We also demonstrate real-time performance across 20 users at a remarkably low latency of 60ms. Our user testing results align with the positive feedback we received during our recent successful demo at AWE-USA-2024.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 17 of its 29 references that are in the map. First posted on arXiv 6 Jul 2024.

Builds on

17 of 29 references.

References · 17

  1. 2019Event-based Vision: A Surveycited by 1,010
  2. 2019Events-to-Video: Bringing Modern Computer Vision to Event Camerascited by 191
  3. 2020Learning to Detect Objects with a 1 Megapixel Event Cameracited by 184
  4. 2022AEGNN: Asynchronous Event-based Graph Neural Networkscited by 118
  5. 2019Graph-Based Object Classification for Neuromorphic Vision Sensingcited by 85
  6. 2023From Chaos Comes Order: Ordering Event Representations for Object Recognition and Detectioncited by 52
  7. 2020EventHands: Real-Time Neural 3D Hand Pose Estimation from an Event Streamcited by 48
  8. 2024Towards Robust Event-guided Low-Light Image Enhancement: A Large-Scale Real-World Event-Image Dataset and Novel Approachcited by 48
  9. 2023Hierarchical Neural Memory Network for Low Latency Event Processingcited by 45
  10. 2024Scene Adaptive Sparse Transformer for Event-based Object Detectioncited by 39
  11. 2024OpenESS: Event-based Semantic Scene Understanding with Open Vocabulariescited by 37
  12. 2024EventEgo3D: 3D Human Motion Capture from Egocentric Event Streamscited by 29
5 more
  1. 2021EventPoint: Self-Supervised Interest Point Detection and Description for Event-based Cameracited by 24
  2. 2023LEOD: Label-Efficient Object Detection for Event Camerascited by 24
  3. 2024A Simple and Effective Point-based Network for Event Camera 6-DOFs Pose Relocalizationcited by 17
  4. 2024SEVD: Synthetic Event-based Vision Dataset for Ego and Fixed Traffic Perceptioncited by 15
  5. 2024Co-designing a Sub-millisecond Latency Event-based Eye Tracking System with Submanifold Sparse CNNcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 5.

8 papers.

  1. 2025EgoEvGesture: Gesture Recognition Based on Egocentric Event Cameracited by 3
  2. 2025Reading in the Dark with Foveated Event Visioncited by 2
  3. 2026EgoEV-HandPose: Egocentric 3D Hand Pose Estimation and Gesture Recognition with Stereo Event Camerascited by 2
  4. 2025Helios 2.0: A Robust, Ultra-Low Power Gesture Recognition System Optimised for Event-Sensor based Wearablescited by 1
  5. 2026Beyond Language: Grounding Referring Expressions with Hand Pointing in Egocentric Visioncited by 1
  6. 2026From Muscle to Text with MyoText: sEMG to Text via Finger Classification and Transformer-Based Decodingcited by 0
  7. 2026OpenGlass: Ultra-Low-Power On-Device AI Eyewear with Event-based Visioncited by 0
  8. 2026AI Smart Glasses for Wearable Intelligence: From Egocentric Sensing to Agentic Personalizationcited by 0