# Robust and Context-Aware Real-Time Collaborative Robot Handling via Dynamic Gesture Commands

arXiv: https://arxiv.org/abs/2304.06175 · HTML: https://mlsanity.com/paper/2304.06175/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Apr 2023.

## Built on by (most cited first, 2 of 2)

- [GUARD: A Safe Reinforcement Learning Benchmark](https://mlsanity.com/paper/2305.13681/) (2023)
- [AssemblyComplete: 3D Combinatorial Construction with Deep Reinforcement Learning](https://mlsanity.com/paper/2410.15469/) (2024)

## Abstract (arXiv)

This paper studies real-time collaborative robot (cobot) handling, where the cobot maneuvers an object under human dynamic gesture commands. Enabling dynamic gesture commands is useful when the human needs to avoid direct contact with the robot or the object handled by the robot. However, the key challenge lies in the heterogeneity in human behaviors and the stochasticity in the perception of dynamic gestures, which requires the robot handling policy to be adaptable and robust. To address these challenges, we introduce Conditional Collaborative Handling Process (CCHP) to encode a contextaware cobot handling policy and a procedure to learn such policy from human-human collaboration. We thoroughly evaluate the adaptability and robustness of CCHP and apply our approach to a real-time cobot assembly task with Kinova Gen3 robot arm. Results show that our method leads to significantly less human effort and smoother human-robot collaboration than state-of-the-art rule-based approach even with first-time users.