# Falsification of Cyber-Physical Systems Using Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1805.00200 · HTML: https://mlsanity.com/paper/1805.00200/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 May 2018.

## Built on by (most cited first, 17 of 17)

- [Reinforcement Learning for Feedback-Enabled Cyber Resilience](https://mlsanity.com/paper/2107.00783/) (2021)
- [MDPFuzz: Testing Models Solving Markov Decision Processes](https://mlsanity.com/paper/2112.02807/) (2021)
- [Finding Failures in High-Fidelity Simulation using Adaptive Stress Testing and the Backward Algorithm](https://mlsanity.com/paper/2107.12940/) (2021)
- [Part-X: A Family of Stochastic Algorithms for Search-Based Test Generation with Probabilistic Guarantees](https://mlsanity.com/paper/2110.10729/) (2021)
- [Diffusion-Based Failure Sampling for Evaluating Safety-Critical Autonomous Systems](https://mlsanity.com/paper/2406.14761/) (2024)
- [Reinforcement Learning for Intelligent Healthcare Systems: A Comprehensive Survey](https://mlsanity.com/paper/2108.04087/) (2021)
- [A Verification Framework for Certifying Learning-Based Safety-Critical Aviation Systems](https://mlsanity.com/paper/2205.04590/) (2022)
- [Model-based Validation as Probabilistic Inference](https://mlsanity.com/paper/2305.09930/) (2023)
- [Requirement falsification for cyber-physical systems using generative models](https://mlsanity.com/paper/2310.20493/) (2023)
- [A Deep Reinforcement Learning Approach to Rare Event Estimation](https://mlsanity.com/paper/2211.12470/) (2022)
- [Security of Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2212.06123/) (2022)
- [Scalable and Interpretable Verification of Image-based Neural Network Controllers for Autonomous Vehicles](https://mlsanity.com/paper/2501.14009/) (2025)
- [Automated System-level Testing of Unmanned Aerial Systems](https://mlsanity.com/paper/2403.15857/) (2024)
- [DiCriTest: Testing Scenario Generation for Decision-Making Agents Considering Diversity and Criticality](https://mlsanity.com/paper/2508.11514/) (2025)
- [Worrisome Properties of Neural Network Controllers and Their Symbolic Representations](https://mlsanity.com/paper/2307.15456/) (2023)
- [Falsification-driven reinforcement learning for maritime motion planning](https://mlsanity.com/paper/2510.06970/) (2025)
- [Validating Generalist Robots with Situation Calculus and STL Falsification](https://mlsanity.com/paper/2601.03038/) (2026)

## Abstract (arXiv)

With the rapid development of software and distributed computing, Cyber-Physical Systems (CPS) are widely adopted in many application areas, e.g., smart grid, autonomous automobile. It is difficult to detect defects in CPS models due to the complexities involved in the software and physical systems. To find defects in CPS models efficiently, robustness guided falsification of CPS is introduced. Existing methods use several optimization techniques to generate counterexamples, which falsify the given properties of a CPS. However those methods may require a large number of simulation runs to find the counterexample and is far from practical. In this work, we explore state-of-the-art Deep Reinforcement Learning (DRL) techniques to reduce the number of simulation runs required to find such counterexamples. We report our method and the preliminary evaluation results.