# Reachable Set Estimation for Neural Network Control Systems: A Simulation-Guided Approach

arXiv: https://arxiv.org/abs/2004.12273 · HTML: https://mlsanity.com/paper/2004.12273/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Apr 2020.

## Built on by (most cited first, 30 of 36)

- [Adversarial Robustness of Deep Neural Networks: A Survey from a Formal Verification Perspective](https://mlsanity.com/paper/2206.12227/) (2022)
- [Reachability Analysis of Neural Feedback Loops](https://mlsanity.com/paper/2108.04140/) (2021)
- [Backward Reachability Analysis of Neural Feedback Loops: Techniques for Linear and Nonlinear Systems](https://mlsanity.com/paper/2209.14076/) (2022)
- [Neural Network Verification in Control](https://mlsanity.com/paper/2110.01388/) (2021)
- [Efficient Reachability Analysis of Closed-Loop Systems with Neural Network Controllers](https://mlsanity.com/paper/2101.01815/) (2021)
- [Backward Reachability Analysis for Neural Feedback Loops](https://mlsanity.com/paper/2204.08319/) (2022)
- [Interval Reachability of Nonlinear Dynamical Systems with Neural Network Controllers](https://mlsanity.com/paper/2301.07912/) (2023)
- [Efficient Interaction-Aware Interval Analysis of Neural Network Feedback Loops](https://mlsanity.com/paper/2307.14938/) (2023)
- [PREMAP: A Unifying PREiMage APproximation Framework for Neural Networks](https://mlsanity.com/paper/2408.09262/) (2024)
- [Provably Safe Neural Network Controllers via Differential Dynamic Logic](https://mlsanity.com/paper/2402.10998/) (2024)
- [A Hybrid Partitioning Strategy for Backward Reachability of Neural Feedback Loops](https://mlsanity.com/paper/2210.07918/) (2022)
- [Reachable Polyhedral Marching (RPM): An Exact Analysis Tool for Deep-Learned Control Systems](https://mlsanity.com/paper/2210.08339/) (2022)
- [Provable Fairness for Neural Network Models using Formal Verification](https://mlsanity.com/paper/2212.08578/) (2022)
- [A Toolbox for Fast Interval Arithmetic in numpy with an Application to Formal Verification of Neural Network Controlled Systems](https://mlsanity.com/paper/2306.15340/) (2023)
- [Reachability analysis of neural networks using mixed monotonicity](https://mlsanity.com/paper/2111.07683/) (2021)
- [Approximate Bisimulation Relations for Neural Networks and Application to Assured Neural Network Compression](https://mlsanity.com/paper/2202.01214/) (2022)
- [Contraction-Guided Adaptive Partitioning for Reachability Analysis of Neural Network Controlled Systems](https://mlsanity.com/paper/2304.03671/) (2023)
- [Verification of Neural Network Control Systems using Symbolic Zonotopes and Polynotopes](https://mlsanity.com/paper/2306.14619/) (2023)
- [Provable Bounds on the Hessian of Neural Networks: Derivative-Preserving Reachability Analysis](https://mlsanity.com/paper/2406.04476/) (2024)
- [TTT: A Temporal Refinement Heuristic for Tenuously Tractable Discrete Time Reachability Problems](https://mlsanity.com/paper/2407.14394/) (2024)
- [Safety Verification of Neural Network Control Systems Using Guaranteed Neural Network Model Reduction](https://mlsanity.com/paper/2301.07531/) (2023)
- [Online Verification of Deep Neural Networks under Domain Shift or Network Updates](https://mlsanity.com/paper/2106.12732/) (2021)
- [Robust Optimization Framework for Training Shallow Neural Networks Using Reachability Method](https://mlsanity.com/paper/2107.12801/) (2021)
- [A Data-Driven Hybrid Automaton Framework to Modeling Complex Dynamical Systems](https://mlsanity.com/paper/2304.13811/) (2023)
- [Learning Verifiable Control Policies Using Relaxed Verification](https://mlsanity.com/paper/2504.16879/) (2025)
- [ECLipsE-Gen-Local: Efficient Compositional Local Lipschitz Estimates for Deep Neural Networks](https://mlsanity.com/paper/2510.05261/) (2025)
- [Failing with Grace: Learning Neural Network Controllers that are Boundedly Unsafe](https://mlsanity.com/paper/2106.11881/) (2021)
- [An Empirical Analysis of the Use of Real-Time Reachability for the Safety Assurance of Autonomous Vehicles](https://mlsanity.com/paper/2205.01419/) (2022)
- [NExG: Provable and Guided State Space Exploration of Neural Network Control Systems using Sensitivity Approximation](https://mlsanity.com/paper/2207.03884/) (2022)
- [Reachability Analysis of Neural Networks with Uncertain Parameters](https://mlsanity.com/paper/2303.07917/) (2023)

## Abstract (arXiv)

The vulnerability of artificial intelligence (AI) and machine learning (ML) against adversarial disturbances and attacks significantly restricts their applicability in safety-critical systems including cyber-physical systems (CPS) equipped with neural network components at various stages of sensing and control. This paper addresses the reachable set estimation and safety verification problems for dynamical systems embedded with neural network components serving as feedback controllers. The closed-loop system can be abstracted in the form of a continuous-time sampled-data system under the control of a neural network controller. First, a novel reachable set computation method in adaptation to simulations generated out of neural networks is developed. The reachability analysis of a class of feedforward neural networks called multilayer perceptrons (MLP) with general activation functions is performed in the framework of interval arithmetic. Then, in combination with reachability methods developed for various dynamical system classes modeled by ordinary differential equations, a recursive algorithm is developed for over-approximating the reachable set of the closed-loop system. The safety verification for neural network control systems can be performed by examining the emptiness of the intersection between the over-approximation of reachable sets and unsafe sets. The effectiveness of the proposed approach has been validated with evaluations on a robotic arm model and an adaptive cruise control system.