# Efficient Sensor Fusion for Gesture Recognition on Resource-Constrained Devices

arXiv: https://arxiv.org/abs/2605.13462 · HTML: https://mlsanity.com/paper/2605.13462/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 24 references that are in the map.
- First posted on arXiv 13 May 2026.

## Builds on (most cited first, 5 of 5)

- [A Machine Learning-oriented Survey on Tiny Machine Learning](https://mlsanity.com/paper/2309.11932/) (2023)
- [Low-latency hand gesture recognition with a low resolution thermal imager](https://mlsanity.com/paper/2004.11623/) (2020)
- [Resource-Efficient Gesture Recognition using Low-Resolution Thermal Camera via Spiking Neural Networks and Sparse Segmentation](https://mlsanity.com/paper/2401.06563/) (2024)
- [Benchmarking Energy and Latency in TinyML: A Novel Method for Resource-Constrained AI](https://mlsanity.com/paper/2505.15622/) (2025)
- [HaGRIDv2: 1M Images for Static and Dynamic Hand Gesture Recognition](https://mlsanity.com/paper/2412.01508/) (2024)

## Abstract (arXiv)

Gesture recognition is a cornerstone of Human-Computer Interaction (HCI) for smart eyewear, enabling natural and device-free control in augmented reality environments. Traditional vision-based approaches face significant challenges regarding power consumption, computational latency, and user privacy. This paper proposes a lightweight, privacy-preserving gesture recognition system based on the fusion of low-resolution Time-of-Flight (ToF) and Infrared (IR) thermal sensors. We used an 8 times 8 multizone ToF sensor (VL53L8CH) and an 8 times 8 IR array (AMG8833) to capture complementary depth and thermal cues. A compact Convolutional Neural Network (CNN) with a specialized grouped-convolution architecture is designed to fuse these modalities efficiently on a microcontroller (MCU). Experimental results on a custom dataset of 7 static gestures, validated via k-fold cross-validation, demonstrate that the proposed fusion strategy significantly outperforms single-sensor baselines with an accuracy of 92.3% and a macro F1-score of 0.93. Finally, on-device benchmarks on STM32F4 and STM32H7 MCUs confirm the system's suitability for resource-constrained wearables, requiring only 6,343 parameters and achieving millisecond-level inference latency with a total system power of 50 mW.