# Including Uncertainty when Learning from Human Corrections

arXiv: https://arxiv.org/abs/1806.02454 · HTML: https://mlsanity.com/paper/1806.02454/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jun 2018.

## Built on by (most cited first, 15 of 15)

- [Active Exploration for Inverse Reinforcement Learning](https://mlsanity.com/paper/2207.08645/) (2022)
- [Learning Reward Functions from Scale Feedback](https://mlsanity.com/paper/2110.00284/) (2021)
- [Dexterous Manipulation from Images: Autonomous Real-World RL via Substep Guidance](https://mlsanity.com/paper/2212.09902/) (2022)
- [REBOOT: Reuse Data for Bootstrapping Efficient Real-World Dexterous Manipulation](https://mlsanity.com/paper/2309.03322/) (2023)
- [Aligning Robot and Human Representations](https://mlsanity.com/paper/2302.01928/) (2023)
- [Contrastive Learning from Exploratory Actions: Leveraging Natural Interactions for Preference Elicitation](https://mlsanity.com/paper/2501.01367/) (2025)
- [From Action Labels to Sets: Rethinking Action Supervision for Imitation Learning from Corrective Feedback](https://mlsanity.com/paper/2502.07645/) (2025)
- [Learning from Physical Human Feedback: An Object-Centric One-Shot Adaptation Method](https://mlsanity.com/paper/2203.04951/) (2022)
- [Interactive Multi-Objective Probabilistic Preference Learning with Soft and Hard Bounds](https://mlsanity.com/paper/2506.21887/) (2025)
- [Learning Online from Corrective Feedback: A Meta-Algorithm for Robotics](https://mlsanity.com/paper/2104.01021/) (2021)
- [PAC Apprenticeship Learning with Bayesian Active Inverse Reinforcement Learning](https://mlsanity.com/paper/2508.03693/) (2025)
- [Learning Gaussian Policies from Corrective Human Feedback](https://mlsanity.com/paper/1903.05216/) (2019)
- [Aligning Robot Representations with Humans](https://mlsanity.com/paper/2205.07882/) (2022)
- [PrioriTouch: Adapting to User Contact Preferences for Whole-Arm Physical Human-Robot Interaction](https://mlsanity.com/paper/2509.18447/) (2025)
- [Rethinking the Implications of Human Feedback for Preference Learning in Human-Robot Collaboration](https://mlsanity.com/paper/2609.13982/) (2026)

## Abstract (arXiv)

It is difficult for humans to efficiently teach robots how to correctly perform a task. One intuitive solution is for the robot to iteratively learn the human's preferences from corrections, where the human improves the robot's current behavior at each iteration. When learning from corrections, we argue that while the robot should estimate the most likely human preferences, it should also know what it does not know, and integrate this uncertainty as it makes decisions. We advance the state-of-the-art by introducing a Kalman filter for learning from corrections: this approach obtains the uncertainty of the estimated human preferences. Next, we demonstrate how the estimate uncertainty can be leveraged for active learning and risk-sensitive deployment. Our results indicate that obtaining and leveraging uncertainty leads to faster learning from human corrections.