# Duo Streamers: A Streaming Gesture Recognition Framework

arXiv: https://arxiv.org/abs/2502.12297 · HTML: https://mlsanity.com/paper/2502.12297/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 46 references that are in the map.
- First posted on arXiv 17 Feb 2025.

## Builds on (most cited first, 17 of 17)

- [River: machine learning for streaming data in Python](https://mlsanity.com/paper/2012.04740/) (2020)
- [Developing Real-time Streaming Transformer Transducer for Speech Recognition on Large-scale Dataset](https://mlsanity.com/paper/2010.11395/) (2020)
- [A Practical Survey on Faster and Lighter Transformers](https://mlsanity.com/paper/2103.14636/) (2021)
- [Predicting the Computational Cost of Deep Learning Models](https://mlsanity.com/paper/1811.11880/) (2018)
- [Real-time Hand Gesture Detection and Classification Using Convolutional Neural Networks](https://mlsanity.com/paper/1901.10323/) (2019)
- [A Comprehensive Survey of Convolutions in Deep Learning: Applications, Challenges, and Future Trends](https://mlsanity.com/paper/2402.15490/) (2024)
- [Efficient Motion Prediction: A Lightweight & Accurate Trajectory Prediction Model With Fast Training and Inference Speed](https://mlsanity.com/paper/2409.16154/) (2024)
- [The Imaginative Generative Adversarial Network: Automatic Data Augmentation for Dynamic Skeleton-Based Hand Gesture and Human Action Recognition](https://mlsanity.com/paper/2105.13061/) (2021)
- [Graph Neural Network for spatiotemporal data: methods and applications](https://mlsanity.com/paper/2306.00012/) (2023)
- [An Advanced Deep Learning Based Three-Stream Hybrid Model for Dynamic Hand Gesture Recognition](https://mlsanity.com/paper/2408.08035/) (2024)
- [Towards Open-World Gesture Recognition](https://mlsanity.com/paper/2401.11144/) (2024)
- [Forearm Ultrasound based Gesture Recognition on Edge](https://mlsanity.com/paper/2409.09915/) (2024)
- [BAT: Boundary aware transducer for memory-efficient and low-latency ASR](https://mlsanity.com/paper/2305.11571/) (2023)
- [Real-Time Hand Gesture Recognition: Integrating Skeleton-Based Data Fusion and Multi-Stream CNN](https://mlsanity.com/paper/2406.15003/) (2024)
- [RingGesture: A Ring-Based Mid-Air Gesture Typing System Powered by a Deep-Learning Word Prediction Framework](https://mlsanity.com/paper/2410.18100/) (2024)
- [Gesture Recognition from Skeleton Data for Intuitive Human-Machine Interaction](https://mlsanity.com/paper/2008.11497/) (2020)
- [Boosting Gesture Recognition with an Automatic Gesture Annotation Framework](https://mlsanity.com/paper/2401.11150/) (2024)

## Abstract (arXiv)

Gesture recognition in resource-constrained scenarios faces significant challenges in achieving high accuracy and low latency. The streaming gesture recognition framework, Duo Streamers, proposed in this paper, addresses these challenges through a three-stage sparse recognition mechanism, an RNN-lite model with an external hidden state, and specialized training and post-processing pipelines, thereby making innovative progress in real-time performance and lightweight design. Experimental results show that Duo Streamers matches mainstream methods in accuracy metrics, while reducing the real-time factor by approximately 92.3%, i.e., delivering a nearly 13-fold speedup. In addition, the framework shrinks parameter counts to 1/38 (idle state) and 1/9 (busy state) compared to mainstream models. In summary, Duo Streamers not only offers an efficient and practical solution for streaming gesture recognition in resource-constrained devices but also lays a solid foundation for extended applications in multimodal and diverse scenarios.