# Data-Driven Abstraction and Synthesis for Stochastic Systems with Unknown Dynamics

arXiv: https://arxiv.org/abs/2508.15543 · HTML: https://mlsanity.com/paper/2508.15543/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Aug 2025.

## Built on by (most cited first, 1 of 1)

- [Robust Shielding for Safe Reinforcement Learning](https://mlsanity.com/paper/2606.00270/) (2026)

## Abstract (arXiv)

We study the automated abstraction-based synthesis of correct-by-construction control policies for stochastic dynamical systems with unknown dynamics. Our approach is to learn an abstraction from sampled data, which is represented in the form of a finite Markov decision process (MDP). In this paper, we present a data-driven technique for constructing finite-state interval MDP (IMDP) abstractions of stochastic systems with unknown nonlinear dynamics. As a distinguishing and novel feature, our technique only requires (1) noisy state-input-state observations and (2) an upper bound on the system's Lipschitz constant. Combined with standard model-checking techniques, our IMDP abstractions enable the synthesis of policies that satisfy probabilistic temporal properties (such as "reach-while-avoid") with a predefined confidence. Our experimental results show the effectiveness and robustness of our approach.