# Learning Certifiably Robust Controllers Using Fragile Perception

arXiv: https://arxiv.org/abs/2209.11328 · HTML: https://mlsanity.com/paper/2209.11328/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Sep 2022.

## Built on by (most cited first, 1 of 1)

- [Safe Perception-Based Control under Stochastic Sensor Uncertainty using Conformal Prediction](https://mlsanity.com/paper/2304.00194/) (2023)

## Abstract (arXiv)

Advances in computer vision and machine learning enable robots to perceive their surroundings in powerful new ways, but these perception modules have well-known fragilities. We consider the problem of synthesizing a safe controller that is robust despite perception errors. The proposed method constructs a state estimator based on Gaussian processes with input-dependent noises. This estimator computes a high-confidence set for the actual state given a perceived state. Then, a robust neural network controller is synthesized that can provably handle the state uncertainty. Furthermore, an adaptive sampling algorithm is proposed to jointly improve the estimator and controller. Simulation experiments, including a realistic vision-based lane-keeping example in CARLA, illustrate the promise of the proposed approach in synthesizing robust controllers with deep-learning-based perception.