# Real-Time Hand Gesture Recognition for OpenXR Using Transformer-Based Machine Learning

arXiv: https://arxiv.org/abs/2609.25466 · HTML: https://mlsanity.com/paper/2609.25466/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 4 of its 8 references that are in the map.
- First posted on arXiv 21 Sep 2026.

## Builds on (most cited first, 4 of 4)

- [Construct Dynamic Graphs for Hand Gesture Recognition via Spatial-Temporal Attention](https://mlsanity.com/paper/1907.08871/) (2019)
- [TraHGR: Transformer for Hand Gesture Recognition via ElectroMyography](https://mlsanity.com/paper/2203.16336/) (2022)
- [GestFormer: Multiscale Wavelet Pooling Transformer Network for Dynamic Hand Gesture Recognition](https://mlsanity.com/paper/2405.11180/) (2024)
- [DSTSA-GCN: Advancing Skeleton-Based Gesture Recognition with Semantic-Aware Spatio-Temporal Topology Modeling](https://mlsanity.com/paper/2501.12086/) (2025)

## Abstract (arXiv)

Hand gesture recognition is a key component in human-computer interaction (HCI), enabling intuitive interfaces for applications in gaming, virtual reality (VR), robotics, and more. This study integrates transformer-based machine-learning models for real-time hand gesture recognition, using hand-tracking data captured through the OpenXR standard in Unity. We leverage positional data of hand joints and wrist rotation angles to train a custom gesture recognition system. By utilizing the sequential modeling capabilities of transformers, the system captures temporal dependencies within short gesture windows and classifies gestures robustly across hand orientations and sizes. The results show a significant improvement in gesture classification accuracy. Building on this, we outline how the approach can be extended toward detecting the flow of movement, i.e., the transitions between gestures, as future work.