# Optimizing Falsification for Learning-Based Control Systems: A Multi-Fidelity Bayesian Approach

arXiv: https://arxiv.org/abs/2409.08097 · HTML: https://mlsanity.com/paper/2409.08097/

## Key facts

- No citing papers in the map yet (data through 7 Oct 2026).
- Builds on 19 of its 52 references that are in the map.
- First posted on arXiv 12 Sep 2024.

## Builds on (most cited first, 19 of 19)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [A Tutorial on Bayesian Optimization](https://mlsanity.com/paper/1807.02811/) (2018)
- [Scalable Global Optimization via Local Bayesian Optimization](https://mlsanity.com/paper/1910.01739/) (2019)
- [Bayesian Optimization in a Billion Dimensions via Random Embeddings](https://mlsanity.com/paper/1301.1942/) (2013)
- [Practical Multi-fidelity Bayesian Optimization for Hyperparameter Tuning](https://mlsanity.com/paper/1903.04703/) (2019)
- [Safe Controller Optimization for Quadrotors with Gaussian Processes](https://mlsanity.com/paper/1509.01066/) (2015)
- [Scalable End-to-End Autonomous Vehicle Testing via Rare-event Simulation](https://mlsanity.com/paper/1811.00145/) (2018)
- [πBO: Augmenting Acquisition Functions with User Beliefs for Bayesian Optimization](https://mlsanity.com/paper/2204.11051/) (2022)
- [Virtual vs. Real: Trading Off Simulations and Physical Experiments in Reinforcement Learning with Bayesian Optimization](https://mlsanity.com/paper/1703.01250/) (2017)
- [Testing Scenario Library Generation for Connected and Automated Vehicles, Part I: Methodology](https://mlsanity.com/paper/1905.03419/) (2019)
- [Safe Learning and Optimization Techniques: Towards a Survey of the State of the Art](https://mlsanity.com/paper/2101.09505/) (2021)
- [Multi-fidelity machine-learning with uncertainty quantification and Bayesian optimization for materials design: Application to ternary random alloys](https://mlsanity.com/paper/2006.00139/) (2020)
- [Verifying Controllers Against Adversarial Examples with Bayesian Optimization](https://mlsanity.com/paper/1802.08678/) (2018)
- [Falsification of Learning-Based Controllers through Multi-Fidelity Bayesian Optimization](https://mlsanity.com/paper/2212.14118/) (2022)
- [Falsification of Cyber-Physical Systems using Bayesian Optimization](https://mlsanity.com/paper/2209.06735/) (2022)
- [BEACON: A Bayesian Evolutionary Approach for Counterexample Generation of Control Systems](https://mlsanity.com/paper/2403.05925/) (2024)
- [A Framework for Controlling Multi-Robot Systems Using Bayesian Optimization and Linear Combination of Vectors](https://mlsanity.com/paper/2203.12416/) (2022)

## Abstract (arXiv)

Testing controllers in safety-critical systems is vital for ensuring their safety and preventing failures. In this paper, we address the falsification problem within learning-based closed-loop control systems through simulation. This problem involves the identification of counterexamples that violate system safety requirements and can be formulated as an optimization task based on these requirements. Using full-fidelity simulator data in this optimization problem can be computationally expensive. To improve efficiency, we propose a multi-fidelity Bayesian optimization falsification framework that harnesses simulators with varying levels of accuracy. Our proposed framework can transition between different simulators and establish meaningful relationships between them. Through multi-fidelity Bayesian optimization, we determine both the optimal system input likely to be a counterexample and the appropriate fidelity level for assessment. We evaluated our approach across various Gym environments, each featuring different levels of fidelity. Our experiments demonstrate that multi-fidelity Bayesian optimization is more computationally efficient than full-fidelity Bayesian optimization and other baseline methods in detecting counterexamples. A Python implementation of the algorithm is available at https://github.com/SAILRIT/MFBO_Falsification.