# Probabilities Are Not Enough: Formal Controller Synthesis for Stochastic Dynamical Models with Epistemic Uncertainty arXiv: https://arxiv.org/abs/2210.05989 ยท HTML: https://mlsanity.com/paper/2210.05989/ ## Key facts - Cited by 11 papers in the map (data through 14 Oct 2026). - Builds on 25 of its 69 references that are in the map. - First posted on arXiv 12 Oct 2022. ## Builds on (most cited first, 25 of 25) - [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019) - [Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning](https://mlsanity.com/paper/2108.06266/) (2021) - [Safe Model-based Reinforcement Learning with Stability Guarantees](https://mlsanity.com/paper/1705.08551/) (2017) - [A General Framework for Uncertainty Estimation in Deep Learning](https://mlsanity.com/paper/1907.06890/) (2019) - [Safe Reinforcement Learning Using Robust MPC](https://mlsanity.com/paper/1906.04005/) (2019) - [Finite Sample Analysis of Stochastic System Identification](https://mlsanity.com/paper/1903.09122/) (2019) - [Reach-Avoid Problems with Time-Varying Dynamics, Targets and Constraints](https://mlsanity.com/paper/1410.6445/) (2014) - [Funnel Libraries for Real-Time Robust Feedback Motion Planning](https://mlsanity.com/paper/1601.04037/) (2016) - [Disentangling Epistemic and Aleatoric Uncertainty in Reinforcement Learning](https://mlsanity.com/paper/2206.01558/) (2022) - [Planning with Learned Dynamics: Probabilistic Guarantees on Safety and Reachability via Lipschitz Constants](https://mlsanity.com/paper/2010.08993/) (2020) - [Learning Control Policies for Stochastic Systems with Reach-avoid Guarantees](https://mlsanity.com/paper/2210.05308/) (2022) - [Safe Optimal Control Using Stochastic Barrier Functions and Deep Forward-Backward SDEs](https://mlsanity.com/paper/2009.01196/) (2020) - [Automated Verification and Synthesis of Stochastic Hybrid Systems: A Survey](https://mlsanity.com/paper/2101.07491/) (2021) - [On the exact feasibility of convex scenario programs with discarded constraints](https://mlsanity.com/paper/2003.13433/) (2020) - [StocHy: automated verification and synthesis of stochastic processes](https://mlsanity.com/paper/1901.10287/) (2019) - [Model Error Propagation via Learned Contraction Metrics for Safe Feedback Motion Planning of Unknown Systems](https://mlsanity.com/paper/2104.08695/) (2021) - [Sampling-Based Robust Control of Autonomous Systems with Non-Gaussian Noise](https://mlsanity.com/paper/2110.12662/) (2021) - [Efficiency through Uncertainty: Scalable Formal Synthesis for Stochastic Hybrid Systems](https://mlsanity.com/paper/1901.01576/) (2019) - [Safety Verification of Unknown Dynamical Systems via Gaussian Process Regression](https://mlsanity.com/paper/2004.01821/) (2020) - [ProbReach: Verified Probabilistic Delta-Reachability for Stochastic Hybrid Systems](https://mlsanity.com/paper/1410.8060/) (2014) - [Constructing MDP Abstractions Using Data with Formal Guarantees](https://mlsanity.com/paper/2206.14402/) (2022) - [Data Driven Stability Analysis of Black-box Switched Linear Systems](https://mlsanity.com/paper/1803.03687/) (2018) - [Correct-by-construction reach-avoid control of partially observable linear stochastic systems](https://mlsanity.com/paper/2103.02398/) (2021) - [Data-driven Abstractions with Probabilistic Guarantees for Linear PETC Systems](https://mlsanity.com/paper/2203.05522/) (2022) - [On-the-fly control of unknown nonlinear systems with sublinear regret](https://mlsanity.com/paper/2206.11103/) (2022) ## Built on by (most cited first, 11 of 11) - [Robust Markov Decision Processes: A Place Where AI and Formal Methods Meet](https://mlsanity.com/paper/2411.11451/) (2024) - [Robust Control for Dynamical Systems With Non-Gaussian Noise via Formal Abstractions](https://mlsanity.com/paper/2301.01526/) (2023) - [Compositional Policy Learning in Stochastic Control Systems with Formal Guarantees](https://mlsanity.com/paper/2312.01456/) (2023) - [Decision-Making Under Uncertainty: Beyond Probabilities](https://mlsanity.com/paper/2303.05848/) (2023) - [NeuroStrata: Harnessing Neurosymbolic Paradigms for Improved Design, Testability, and Verifiability of Autonomous CPS](https://mlsanity.com/paper/2502.12267/) (2025) - [Data-Driven Abstractions via Binary-Tree Gaussian Processes for Formal Verification](https://mlsanity.com/paper/2407.21029/) (2024) - [Certified Policy Verification and Synthesis for MDPs under Distributional Reach-avoidance Properties](https://mlsanity.com/paper/2405.04015/) (2024) - [Formal Controller Synthesis for Markov Jump Linear Systems with Uncertain Dynamics](https://mlsanity.com/paper/2212.00679/) (2022) - [Formal Control for Uncertain Systems via Contract-Based Probabilistic Surrogates (Extended Version)](https://mlsanity.com/paper/2506.16971/) (2025) - [Correct-by-Construction Control for Stochastic and Uncertain Dynamical Models via Formal Abstractions](https://mlsanity.com/paper/2311.09786/) (2023) - [Correcting Learning-based Perception for Safety](https://mlsanity.com/paper/2609.22108/) (2026) ## Abstract (arXiv) Capturing uncertainty in models of complex dynamical systems is crucial to designing safe controllers. Stochastic noise causes aleatoric uncertainty, whereas imprecise knowledge of model parameters leads to epistemic uncertainty. Several approaches use formal abstractions to synthesize policies that satisfy temporal specifications related to safety and reachability. However, the underlying models exclusively capture aleatoric but not epistemic uncertainty, and thus require that model parameters are known precisely. Our contribution to overcoming this restriction is a novel abstraction-based controller synthesis method for continuous-state models with stochastic noise and uncertain parameters. By sampling techniques and robust analysis, we capture both aleatoric and epistemic uncertainty, with a user-specified confidence level, in the transition probability intervals of a so-called interval Markov decision process (iMDP). We synthesize an optimal policy on this iMDP, which translates (with the specified confidence level) to a feedback controller for the continuous model with the same performance guarantees. Our experimental benchmarks confirm that accounting for epistemic uncertainty leads to controllers that are more robust against variations in parameter values.