# Gesture2Path: Imitation Learning for Gesture-aware Navigation

arXiv: https://arxiv.org/abs/2209.09375 · HTML: https://mlsanity.com/paper/2209.09375/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Social robot navigation, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 8 of its 45 references that are in the map.
- First posted on arXiv 19 Sep 2022.

## Placed in

- Offline RL and imitation learning from data / [Social robot navigation](https://mlsanity.com/offlinerl/social-robot-navigation/)

## Builds on (most cited first, 8 of 8)

- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Omnivore: A Single Model for Many Visual Modalities](https://mlsanity.com/paper/2201.08377/) (2022)
- [Socially Compliant Navigation Dataset (SCAND): A Large-Scale Dataset of Demonstrations for Social Navigation](https://mlsanity.com/paper/2203.15041/) (2022)
- [Core Challenges of Social Robot Navigation: A Survey](https://mlsanity.com/paper/2103.05668/) (2021)
- [Learning an Approximate Model Predictive Controller with Guarantees](https://mlsanity.com/paper/1806.04167/) (2018)
- [Long-Range Indoor Navigation with PRM-RL](https://mlsanity.com/paper/1902.09458/) (2019)
- [A Protocol for Validating Social Navigation Policies](https://mlsanity.com/paper/2204.05443/) (2022)

## Built on by (most cited first, 3 of 3)

- [Principles and Guidelines for Evaluating Social Robot Navigation Algorithms](https://mlsanity.com/paper/2306.16740/) (2023)
- [Advancing Social Intelligence in AI Agents: Technical Challenges and Open Questions](https://mlsanity.com/paper/2404.11023/) (2024)
- [Probing Prompt Design for Socially Compliant Robot Navigation with Vision Language Models](https://mlsanity.com/paper/2601.14622/) (2026)

## Abstract (arXiv)

As robots increasingly enter human-centered environments, they must not only be able to navigate safely around humans, but also adhere to complex social norms. Humans often rely on non-verbal communication through gestures and facial expressions when navigating around other people, especially in densely occupied spaces. Consequently, robots also need to be able to interpret gestures as part of solving social navigation tasks. To this end, we present Gesture2Path, a novel social navigation approach that combines image-based imitation learning with model-predictive control. Gestures are interpreted based on a neural network that operates on streams of images, while we use a state-of-the-art model predictive control algorithm to solve point-to-point navigation tasks. We deploy our method on real robots and showcase the effectiveness of our approach for the four gestures-navigation scenarios: left/right, follow me, and make a circle. Our experiments indicate that our method is able to successfully interpret complex human gestures and to use them as a signal to generate socially compliant trajectories for navigation tasks. We validated our method based on in-situ ratings of participants interacting with the robots.