# Physical Interaction as Communication: Learning Robot Objectives Online from Human Corrections

arXiv: https://arxiv.org/abs/2107.02349 · HTML: https://mlsanity.com/paper/2107.02349/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 62 references that are in the map.
- First posted on arXiv 6 Jul 2021.

## Builds on (most cited first, 12 of 12)

- [End-to-End Training of Deep Visuomotor Policies](https://mlsanity.com/paper/1504.00702/) (2015)
- [An Algorithmic Perspective on Imitation Learning](https://mlsanity.com/paper/1811.06711/) (2018)
- [Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization](https://mlsanity.com/paper/1603.00448/) (2016)
- [Cooperative Inverse Reinforcement Learning](https://mlsanity.com/paper/1606.03137/) (2016)
- [DESPOT: Online POMDP Planning with Regularization](https://mlsanity.com/paper/1609.03250/) (2016)
- [Shared Autonomy via Hindsight Optimization for Teleoperation and Teaming](https://mlsanity.com/paper/1706.00155/) (2017)
- [Learning Preferences for Manipulation Tasks from Online Coactive Feedback](https://mlsanity.com/paper/1601.00741/) (2016)
- [Learning Human Objectives from Sequences of Physical Corrections](https://mlsanity.com/paper/2104.00078/) (2021)
- [Human-Robot Mutual Adaptation in Shared Autonomy](https://mlsanity.com/paper/1701.07851/) (2017)
- [I Know What You Meant: Learning Human Objectives by (Under)estimating Their Choice Set](https://mlsanity.com/paper/2011.06118/) (2020)
- [Trajectory Deformations from Physical Human-Robot Interaction](https://mlsanity.com/paper/1710.09871/) (2017)
- [Autonomous Interpretation of Demonstrations for Modification of Dynamical Movement Primitives](https://mlsanity.com/paper/1905.11130/) (2019)

## Built on by (most cited first, 19 of 19)

- [Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2307.15217/) (2023)
- [Affordances from Human Videos as a Versatile Representation for Robotics](https://mlsanity.com/paper/2304.08488/) (2023)
- [RT-H: Action Hierarchies Using Language](https://mlsanity.com/paper/2403.01823/) (2024)
- [Inference-Time Policy Steering through Human Interactions](https://mlsanity.com/paper/2411.16627/) (2024)
- [ROS-LLM: A ROS framework for embodied AI with task feedback and structured reasoning](https://mlsanity.com/paper/2406.19741/) (2024)
- [Unified Learning from Demonstrations, Corrections, and Preferences during Physical Human-Robot Interaction](https://mlsanity.com/paper/2207.03395/) (2022)
- [Towards Reliable Advertising Image Generation Using Human Feedback](https://mlsanity.com/paper/2408.00418/) (2024)
- [Autonomous Robotic Reinforcement Learning with Asynchronous Human Feedback](https://mlsanity.com/paper/2310.20608/) (2023)
- [Bootstrapping Linear Models for Fast Online Adaptation in Human-Agent Collaboration](https://mlsanity.com/paper/2404.10733/) (2024)
- [Counterfactual Behavior Cloning: Offline Imitation Learning from Imperfect Human Demonstrations](https://mlsanity.com/paper/2505.10760/) (2025)
- [Leveraging Human Revisions for Improving Text-to-Layout Models](https://mlsanity.com/paper/2405.13026/) (2024)
- [Coaching a Robotic Sonographer: Learning Robotic Ultrasound with Sparse Expert's Feedback](https://mlsanity.com/paper/2409.02337/) (2024)
- [User-Feedback-Driven Adaptation for Vision-and-Language Navigation](https://mlsanity.com/paper/2512.10322/) (2025)
- [Enhancing Goal Inference via Correction Timing](https://mlsanity.com/paper/2602.18603/) (2026)
- [Multi-Modal, Multi-Environment Machine Teaching for Robust Reward Learning](https://mlsanity.com/paper/2607.08647/) (2026)
- [Robots Influencing Humans to Reveal their Goals during Collaboration and Competition](https://mlsanity.com/paper/2609.05519/) (2026)
- [PrioriTouch: Adapting to User Contact Preferences for Whole-Arm Physical Human-Robot Interaction](https://mlsanity.com/paper/2509.18447/) (2025)
- [QuickLAP: Quick Language-Action Preference Learning for Semi-Autonomous Agents](https://mlsanity.com/paper/2511.17855/) (2025)
- [MAVRL: Learning Reward Functions from Multiple Feedback Types with Amortized Variational Inference](https://mlsanity.com/paper/2602.15206/) (2026)

## Abstract (arXiv)

When a robot performs a task next to a human, physical interaction is inevitable: the human might push, pull, twist, or guide the robot. The state-of-the-art treats these interactions as disturbances that the robot should reject or avoid. At best, these robots respond safely while the human interacts; but after the human lets go, these robots simply return to their original behavior. We recognize that physical human-robot interaction (pHRI) is often intentional -- the human intervenes on purpose because the robot is not doing the task correctly. In this paper, we argue that when pHRI is intentional it is also informative: the robot can leverage interactions to learn how it should complete the rest of its current task even after the person lets go. We formalize pHRI as a dynamical system, where the human has in mind an objective function they want the robot to optimize, but the robot does not get direct access to the parameters of this objective -- they are internal to the human. Within our proposed framework human interactions become observations about the true objective. We introduce approximations to learn from and respond to pHRI in real-time. We recognize that not all human corrections are perfect: often users interact with the robot noisily, and so we improve the efficiency of robot learning from pHRI by reducing unintended learning. Finally, we conduct simulations and user studies on a robotic manipulator to compare our proposed approach to the state-of-the-art. Our results indicate that learning from pHRI leads to better task performance and improved human satisfaction.