# Probabilistically Safe Robot Planning with Confidence-Based Human Predictions

arXiv: https://arxiv.org/abs/1806.00109 · HTML: https://mlsanity.com/paper/1806.00109/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 May 2018.

## Built on by (most cited first, 28 of 28)

- [Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformer](https://mlsanity.com/paper/2207.14024/) (2022)
- [The Safety Filter: A Unified View of Safety-Critical Control in Autonomous Systems](https://mlsanity.com/paper/2309.05837/) (2023)
- [The Effect of Modeling Human Rationality Level on Learning Rewards from Multiple Feedback Types](https://mlsanity.com/paper/2208.10687/) (2022)
- [Inducing Structure in Reward Learning by Learning Features](https://mlsanity.com/paper/2201.07082/) (2022)
- [Situational Confidence Assistance for Lifelong Shared Autonomy](https://mlsanity.com/paper/2104.06556/) (2021)
- [Active Uncertainty Reduction for Safe and Efficient Interaction Planning: A Shielding-Aware Dual Control Approach](https://mlsanity.com/paper/2302.00171/) (2023)
- [SHARP: Shielding-Aware Robust Planning for Safe and Efficient Human-Robot Interaction](https://mlsanity.com/paper/2110.00843/) (2021)
- [Conformal Methods for Quantifying Uncertainty in Spatiotemporal Data: A Survey](https://mlsanity.com/paper/2209.03580/) (2022)
- [On complementing end-to-end human behavior predictors with planning](https://mlsanity.com/paper/2103.05661/) (2021)
- [Active Uncertainty Reduction for Human-Robot Interaction: An Implicit Dual Control Approach](https://mlsanity.com/paper/2202.07720/) (2022)
- [Analyzing Human Models that Adapt Online](https://mlsanity.com/paper/2103.05746/) (2021)
- [Efficient Adaptation in Mixed-Motive Environments via Hierarchical Opponent Modeling and Planning](https://mlsanity.com/paper/2406.08002/) (2024)
- [Confidence Composition for Monitors of Verification Assumptions](https://mlsanity.com/paper/2111.03782/) (2021)
- [Blending Data-Driven Priors in Dynamic Games](https://mlsanity.com/paper/2402.14174/) (2024)
- [A Unifying Bayesian Formulation of Measures of Interpretability in Human-AI](https://mlsanity.com/paper/2104.10743/) (2021)
- [Hypergraph-Transformer (HGT) for Interactive Event Prediction in Laparoscopic and Robotic Surgery](https://mlsanity.com/paper/2402.01974/) (2024)
- [Trends in Motion Prediction Toward Deployable and Generalizable Autonomy: A Revisit and Perspectives](https://mlsanity.com/paper/2505.09074/) (2025)
- [Recursively Feasible Shrinking-Horizon MPC in Dynamic Environments with Conformal Prediction Guarantees](https://mlsanity.com/paper/2405.10875/) (2024)
- [Goal Inference from Open-Ended Dialog](https://mlsanity.com/paper/2410.13957/) (2024)
- [A Mental Model Based Framework of Trust](https://mlsanity.com/paper/2301.12569/) (2023)
- [Multi-Agent Strategy Explanations for Human-Robot Collaboration](https://mlsanity.com/paper/2311.11955/) (2023)
- [Stranger Danger! Identifying and Avoiding Unpredictable Pedestrians in RL-based Social Robot Navigation](https://mlsanity.com/paper/2407.06056/) (2024)
- [Context-Based Echo State Networks with Prediction Confidence for Human-Robot Shared Control](https://mlsanity.com/paper/2412.00541/) (2024)
- [From Behavioral Theories to Econometrics: Inferring Preferences of Human Agents from Data on Repeated Interactions](https://mlsanity.com/paper/2112.15151/) (2021)
- [Robots that Suggest Safe Alternatives](https://mlsanity.com/paper/2409.09883/) (2024)
- [Multi-Cycle Spatio-Temporal Adaptation in Human-Robot Teaming](https://mlsanity.com/paper/2604.19670/) (2026)
- [Robots That Know What to Ask: Recovering Misaligned Rewards through Targeted Explanations](https://mlsanity.com/paper/2605.22986/) (2026)
- [Computationally efficient safe exploration in reinforcement learning](https://mlsanity.com/paper/2609.22919/) (2026)

## Abstract (arXiv)

In order to safely operate around humans, robots can employ predictive models of human motion. Unfortunately, these models cannot capture the full complexity of human behavior and necessarily introduce simplifying assumptions. As a result, predictions may degrade whenever the observed human behavior departs from the assumed structure, which can have negative implications for safety. In this paper, we observe that how "rational" human actions appear under a particular model can be viewed as an indicator of that model's ability to describe the human's current motion. By reasoning about this model confidence in a real-time Bayesian framework, we show that the robot can very quickly modulate its predictions to become more uncertain when the model performs poorly. Building on recent work in provably-safe trajectory planning, we leverage these confidence-aware human motion predictions to generate assured autonomous robot motion. Our new analysis combines worst-case tracking error guarantees for the physical robot with probabilistic time-varying human predictions, yielding a quantitative, probabilistic safety certificate. We demonstrate our approach with a quadcopter navigating around a human.