# Online Update of Safety Assurances Using Confidence-Based Predictions

arXiv: https://arxiv.org/abs/2210.01199 · HTML: https://mlsanity.com/paper/2210.01199/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Oct 2022.

## Built on by (most cited first, 15 of 15)

- [The Safety Filter: A Unified View of Safety-Critical Control in Autonomous Systems](https://mlsanity.com/paper/2309.05837/) (2023)
- [Targeted Adversarial Attacks against Neural Network Trajectory Predictors](https://mlsanity.com/paper/2212.04138/) (2022)
- [Deception Game: Closing the Safety-Learning Loop in Interactive Robot Autonomy](https://mlsanity.com/paper/2309.01267/) (2023)
- [How Safe Am I Given What I See? Calibrated Prediction of Safety Chances for Image-Controlled Autonomy](https://mlsanity.com/paper/2308.12252/) (2023)
- [Human-AI Safety: A Descendant of Generative AI and Control Systems Safety](https://mlsanity.com/paper/2405.09794/) (2024)
- [Efficient Adaptation in Mixed-Motive Environments via Hierarchical Opponent Modeling and Planning](https://mlsanity.com/paper/2406.08002/) (2024)
- [Learning Maximal Safe Sets Using Hypernetworks for MPC-based Local Trajectory Planning in Unknown Environments](https://mlsanity.com/paper/2410.20267/) (2024)
- [Physically Interpretable World Models via Weakly Supervised Representation Learning](https://mlsanity.com/paper/2412.12870/) (2024)
- [Reachability Barrier Networks: Learning Hamilton-Jacobi Solutions for Smooth and Flexible Control Barrier Functions](https://mlsanity.com/paper/2505.11755/) (2025)
- [Residual Neural Terminal Constraint for MPC-based Collision Avoidance in Dynamic Environments](https://mlsanity.com/paper/2508.03428/) (2025)
- [The Case for Negative Data: From Crash Reports to Counterfactuals for Reasonable Driving](https://mlsanity.com/paper/2509.18626/) (2025)
- [How Safe Will I Be Given What I Saw? Calibrated Prediction of Safety Chances for Image-Controlled Autonomy](https://mlsanity.com/paper/2508.09346/) (2025)
- [Amelia: A Large Dataset and Benchmark for Airport Surface Movement Forecasting](https://mlsanity.com/paper/2407.21185/) (2024)
- [Robust Planning for Autonomous Driving via Mixed Adversarial Diffusion Predictions](https://mlsanity.com/paper/2505.12327/) (2025)
- [CoCoNav: Conformal Control for Safe Robot Navigation in Crowds](https://mlsanity.com/paper/2608.07751/) (2026)

## Abstract (arXiv)

Robots such as autonomous vehicles and assistive manipulators are increasingly operating in dynamic environments and close physical proximity to people. In such scenarios, the robot can leverage a human motion predictor to predict their future states and plan safe and efficient trajectories. However, no model is ever perfect -- when the observed human behavior deviates from the model predictions, the robot might plan unsafe maneuvers. Recent works have explored maintaining a confidence parameter in the human model to overcome this challenge, wherein the predicted human actions are tempered online based on the likelihood of the observed human action under the prediction model. This has opened up a new research challenge, i.e., textithow to compute the future human states online as the confidence parameter changes? In this work, we propose a Hamilton-Jacobi (HJ) reachability-based approach to overcome this challenge. Treating the confidence parameter as a virtual state in the system, we compute a parameter-conditioned forward reachable tube (FRT) that provides the future human states as a function of the confidence parameter. Online, as the confidence parameter changes, we can simply query the corresponding FRT, and use it to update the robot plan. Computing parameter-conditioned FRT corresponds to an (offline) high-dimensional reachability problem, which we solve by leveraging recent advances in data-driven reachability analysis. Overall, our framework enables online maintenance and updates of safety assurances in human-robot interaction scenarios, even when the human prediction model is incorrect. We demonstrate our approach in several safety-critical autonomous driving scenarios, involving a state-of-the-art deep learning-based prediction model.