# Model Predictive Control for Fluid Human-to-Robot Handovers

arXiv: https://arxiv.org/abs/2204.00134 · HTML: https://mlsanity.com/paper/2204.00134/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 30 references that are in the map.
- First posted on arXiv 31 Mar 2022.

## Builds on (most cited first, 16 of 16)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [6-DOF GraspNet: Variational Grasp Generation for Object Manipulation](https://mlsanity.com/paper/1905.10520/) (2019)
- [Closing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approach](https://mlsanity.com/paper/1804.05172/) (2018)
- [STORM: An Integrated Framework for Fast Joint-Space Model-Predictive Control for Reactive Manipulation](https://mlsanity.com/paper/2104.13542/) (2021)
- [Riemannian Motion Policies](https://mlsanity.com/paper/1801.02854/) (2018)
- [Object Handovers: a Review for Robotics](https://mlsanity.com/paper/2007.12952/) (2020)
- [Reactive Human-to-Robot Handovers of Arbitrary Objects](https://mlsanity.com/paper/2011.08961/) (2020)
- [Object-Independent Human-to-Robot Handovers using Real Time Robotic Vision](https://mlsanity.com/paper/2006.01797/) (2020)
- [Dynamic Grasping with Reachability and Motion Awareness](https://mlsanity.com/paper/2103.10562/) (2021)
- [Human Grasp Classification for Reactive Human-to-Robot Handovers](https://mlsanity.com/paper/2003.06000/) (2020)
- [Relaxed-Rigidity Constraints: Kinematic Trajectory Optimization and Collision Avoidance for In-Grasp Manipulation](https://mlsanity.com/paper/1806.00942/) (2018)
- [Newton methods for k-order Markov Constrained Motion Problems](https://mlsanity.com/paper/1407.0414/) (2014)
- [Workspace Aware Online Grasp Planning](https://mlsanity.com/paper/1806.11402/) (2018)
- [Model Predictive Robot-Environment Interaction Control for Mobile Manipulation Tasks](https://mlsanity.com/paper/2106.04202/) (2021)

## Built on by (most cited first, 9 of 9)

- [Learning Human-to-Robot Handovers from Point Clouds](https://mlsanity.com/paper/2303.17592/) (2023)
- [SynH2R: Synthesizing Hand-Object Motions for Learning Human-to-Robot Handovers](https://mlsanity.com/paper/2311.05599/) (2023)
- [InteRACT: Transformer Models for Human Intent Prediction Conditioned on Robot Actions](https://mlsanity.com/paper/2311.12943/) (2023)
- [RGB-Only Reconstruction of Tabletop Scenes for Collision-Free Manipulator Control](https://mlsanity.com/paper/2210.11668/) (2022)
- [ManiCast: Collaborative Manipulation with Cost-Aware Human Forecasting](https://mlsanity.com/paper/2310.13258/) (2023)
- [Stereo Hand-Object Reconstruction for Human-to-Robot Handover](https://mlsanity.com/paper/2412.07487/) (2024)
- [MINT-RVAE: Multi-Cues Intention Prediction of Human-Robot Interaction using Human Pose and Emotion Information from RGB-only Camera Data](https://mlsanity.com/paper/2509.22573/) (2025)
- [Real-Time Human-Robot Interaction Intent Detection Using RGB-based Pose and Emotion Cues with Cross-Camera Model Generalization](https://mlsanity.com/paper/2512.17958/) (2025)
- [3HANDS Dataset: Learning from Humans for Generating Naturalistic Handovers with Supernumerary Robotic Limbs](https://mlsanity.com/paper/2503.04635/) (2025)

## Abstract (arXiv)

Human-robot handover is a fundamental yet challenging task in human-robot interaction and collaboration. Recently, remarkable progressions have been made in human-to-robot handovers of unknown objects by using learning-based grasp generators. However, how to responsively generate smooth motions to take an object from a human is still an open question. Specifically, planning motions that take human comfort into account is not a part of the human-robot handover process in most prior works. In this paper, we propose to generate smooth motions via an efficient model-predictive control (MPC) framework that integrates perception and complex domain-specific constraints into the optimization problem. We introduce a learning-based grasp reachability model to select candidate grasps which maximize the robot's manipulability, giving it more freedom to satisfy these constraints. Finally, we integrate a neural net force/torque classifier that detects contact events from noisy data. We conducted human-to-robot handover experiments on a diverse set of objects with several users (N=4) and performed a systematic evaluation of each module. The study shows that the users preferred our MPC approach over the baseline system by a large margin. More results and videos are available at https://sites.google.com/nvidia.com/mpc-for-handover.