# Improved Optical Flow for Gesture-based Human-robot Interaction

arXiv: https://arxiv.org/abs/1905.08685 · HTML: https://mlsanity.com/paper/1905.08685/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 May 2019.

## Built on by (most cited first, 8 of 8)

- [FastFlowNet: A Lightweight Network for Fast Optical Flow Estimation](https://mlsanity.com/paper/2103.04524/) (2021)
- [Ultra-Range Gesture Recognition using a Web-Camera in Human-Robot Interaction](https://mlsanity.com/paper/2311.15361/) (2023)
- [NatSGD: A Dataset with Speech, Gestures, and Demonstrations for Robot Learning in Natural Human-Robot Interaction](https://mlsanity.com/paper/2403.02274/) (2024)
- [Fast Monocular Hand Pose Estimation on Embedded Systems](https://mlsanity.com/paper/2102.07067/) (2021)
- [Hand Gesture Recognition Based on a Nonconvex Regularization](https://mlsanity.com/paper/2104.14349/) (2021)
- [Body Gesture Recognition to Control a Social Robot](https://mlsanity.com/paper/2206.07538/) (2022)
- [CompactFlowNet: Efficient Real-time Optical Flow Estimation on Mobile Devices](https://mlsanity.com/paper/2412.13273/) (2024)
- [NatSGLD: A Dataset with Speech, Gesture, Logic, and Demonstration for Robot Learning in Natural Human-Robot Interaction](https://mlsanity.com/paper/2502.16718/) (2025)

## Abstract (arXiv)

Gesture interaction is a natural way of communicating with a robot as an alternative to speech. Gesture recognition methods leverage optical flow in order to understand human motion. However, while accurate optical flow estimation (i.e., traditional) methods are costly in terms of runtime, fast estimation (i.e., deep learning) methods' accuracy can be improved. In this paper, we present a pipeline for gesture-based human-robot interaction that uses a novel optical flow estimation method in order to achieve an improved speed-accuracy trade-off. Our optical flow estimation method introduces four improvements to previous deep learning-based methods: strong feature extractors, attention to contours, midway features, and a combination of these three. This results in a better understanding of motion, and a finer representation of silhouettes. In order to evaluate our pipeline, we generated our own dataset, MIBURI, which contains gestures to command a house service robot. In our experiments, we show how our method improves not only optical flow estimation, but also gesture recognition, offering a speed-accuracy trade-off more realistic for practical robot applications.