# How Should a Robot Assess Risk? Towards an Axiomatic Theory of Risk in Robotics

arXiv: https://arxiv.org/abs/1710.11040 · HTML: https://mlsanity.com/paper/1710.11040/

## Key facts

- Cited by 59 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Oct 2017.

## Built on by (most cited first, 30 of 59)

- [RMIX: Learning Risk-Sensitive Policies for Cooperative Reinforcement Learning Agents](https://mlsanity.com/paper/2102.08159/) (2021)
- [STEP: Stochastic Traversability Evaluation and Planning for Risk-Aware Off-road Navigation](https://mlsanity.com/paper/2103.02828/) (2021)
- [RoadRunner -- Learning Traversability Estimation for Autonomous Off-road Driving](https://mlsanity.com/paper/2402.19341/) (2024)
- [One Risk to Rule Them All: A Risk-Sensitive Perspective on Model-Based Offline Reinforcement Learning](https://mlsanity.com/paper/2212.00124/) (2022)
- [EVORA: Deep Evidential Traversability Learning for Risk-Aware Off-Road Autonomy](https://mlsanity.com/paper/2311.06234/) (2023)
- [Risk-Aware Off-Road Navigation via a Learned Speed Distribution Map](https://mlsanity.com/paper/2203.13429/) (2022)
- [Ensemble Quantile Networks: Uncertainty-Aware Reinforcement Learning with Applications in Autonomous Driving](https://mlsanity.com/paper/2105.10266/) (2021)
- [Risk-Conditioned Distributional Soft Actor-Critic for Risk-Sensitive Navigation](https://mlsanity.com/paper/2104.03111/) (2021)
- [Learning Risk-Aware Costmaps via Inverse Reinforcement Learning for Off-Road Navigation](https://mlsanity.com/paper/2302.00134/) (2023)
- [Learning Risk-Aware Quadrupedal Locomotion using Distributional Reinforcement Learning](https://mlsanity.com/paper/2309.14246/) (2023)
- [Addressing Inherent Uncertainty: Risk-Sensitive Behavior Generation for Automated Driving using Distributional Reinforcement Learning](https://mlsanity.com/paper/2102.03119/) (2021)
- [Conformal Quantitative Predictive Monitoring of STL Requirements for Stochastic Processes](https://mlsanity.com/paper/2211.02375/) (2022)
- [Learning Risk-aware Costmaps for Traversability in Challenging Environments](https://mlsanity.com/paper/2107.11722/) (2021)
- [Risk-averse autonomous systems: A brief history and recent developments from the perspective of optimal control](https://mlsanity.com/paper/2109.08947/) (2021)
- [RAP: Risk-Aware Prediction for Robust Planning](https://mlsanity.com/paper/2210.01368/) (2022)
- [Risk-Averse Model Uncertainty for Distributionally Robust Safe Reinforcement Learning](https://mlsanity.com/paper/2301.12593/) (2023)
- [Risk-Averse Stochastic Shortest Path Planning](https://mlsanity.com/paper/2103.14727/) (2021)
- [Two steps to risk sensitivity](https://mlsanity.com/paper/2111.06803/) (2021)
- [CVaR-based Flight Energy Risk Assessment for Multirotor UAVs using a Deep Energy Model](https://mlsanity.com/paper/2105.15189/) (2021)
- [Adaptive Risk-Tendency: Nano Drone Navigation in Cluttered Environments with Distributional Reinforcement Learning](https://mlsanity.com/paper/2203.14749/) (2022)
- [Risk Verification of Stochastic Systems with Neural Network Controllers](https://mlsanity.com/paper/2209.09881/) (2022)
- [Q-learning for Quantile MDPs: A Decomposition, Performance, and Convergence Analysis](https://mlsanity.com/paper/2410.24128/) (2024)
- [Beyond CVaR: Leveraging Static Spectral Risk Measures for Enhanced Decision-Making in Distributional Reinforcement Learning](https://mlsanity.com/paper/2501.02087/) (2025)
- [Planning for Risk-Aversion and Expected Value in MDPs](https://mlsanity.com/paper/2110.12746/) (2021)
- [Risk-aware Path Planning via Probabilistic Fusion of Traversability Prediction for Planetary Rovers on Heterogeneous Terrains](https://mlsanity.com/paper/2303.01169/) (2023)
- [Safe multi-agent motion planning under uncertainty for drones using filtered reinforcement learning](https://mlsanity.com/paper/2311.00063/) (2023)
- [Emergent Risk Awareness in Rational Agents under Resource Constraints](https://mlsanity.com/paper/2505.23436/) (2025)
- [STL Robustness Risk over Discrete-Time Stochastic Processes](https://mlsanity.com/paper/2104.01503/) (2021)
- [Risk-Driven Design of Perception Systems](https://mlsanity.com/paper/2205.10677/) (2022)
- [Adaptive Failure Search Using Critical States from Domain Experts](https://mlsanity.com/paper/2304.00365/) (2023)

## Abstract (arXiv)

Endowing robots with the capability of assessing risk and making risk-aware decisions is widely considered a key step toward ensuring safety for robots operating under uncertainty. But, how should a robot quantify risk? A natural and common approach is to consider the framework whereby costs are assigned to stochastic outcomes - an assignment captured by a cost random variable. Quantifying risk then corresponds to evaluating a risk metric, i.e., a mapping from the cost random variable to a real number. Yet, the question of what constitutes a "good" risk metric has received little attention within the robotics community. The goal of this paper is to explore and partially address this question by advocating axioms that risk metrics in robotics applications should satisfy in order to be employed as rational assessments of risk. We discuss general representation theorems that precisely characterize the class of metrics that satisfy these axioms (referred to as distortion risk metrics), and provide instantiations that can be used in applications. We further discuss pitfalls of commonly used risk metrics in robotics, and discuss additional properties that one must consider in sequential decision making tasks. Our hope is that the ideas presented here will lead to a foundational framework for quantifying risk (and hence safety) in robotics applications.