# Efficient Determination of Safety Requirements for Perception Systems

arXiv: https://arxiv.org/abs/2307.01371 · HTML: https://mlsanity.com/paper/2307.01371/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 26 references that are in the map.
- First posted on arXiv 3 Jul 2023.

## Builds on (most cited first, 10 of 10)

- [A Tutorial on Bayesian Optimization](https://mlsanity.com/paper/1807.02811/) (2018)
- [An optimal algorithm for the Thresholding Bandit Problem](https://mlsanity.com/paper/1605.08671/) (2016)
- [Truncated Variance Reduction: A Unified Approach to Bayesian Optimization and Level-Set Estimation](https://mlsanity.com/paper/1610.07379/) (2016)
- [Bayesian Algorithm Execution: Estimating Computable Properties of Black-box Functions Using Mutual Information](https://mlsanity.com/paper/2104.09460/) (2021)
- [Robust Super-Level Set Estimation using Gaussian Processes](https://mlsanity.com/paper/1811.09977/) (2018)
- [Bayesian Experimental Design for Finding Reliable Level Set under Input Uncertainty](https://mlsanity.com/paper/1910.12043/) (2019)
- [Active learning for distributionally robust level-set estimation](https://mlsanity.com/paper/2102.04000/) (2021)
- [AirTrack: Onboard Deep Learning Framework for Long-Range Aircraft Detection and Tracking](https://mlsanity.com/paper/2209.12849/) (2022)
- [A Bandit Approach to Multiple Testing with False Discovery Control](https://mlsanity.com/paper/1809.02235/) (2018)
- [Risk-Driven Design of Perception Systems](https://mlsanity.com/paper/2205.10677/) (2022)

## Built on by (most cited first, 1 of 1)

- [Aircraft Collision Avoidance Systems: Technological Challenges and Solutions on the Path to Regulatory Acceptance](https://mlsanity.com/paper/2510.20916/) (2025)

## Abstract (arXiv)

Perception systems operate as a subcomponent of the general autonomy stack, and perception system designers often need to optimize performance characteristics while maintaining safety with respect to the overall closed-loop system. For this reason, it is useful to distill high-level safety requirements into component-level requirements on the perception system. In this work, we focus on efficiently determining sets of safe perception system performance characteristics given a black-box simulator of the fully-integrated, closed-loop system. We combine the advantages of common black-box estimation techniques such as Gaussian processes and threshold bandits to develop a new estimation method, which we call smoothing bandits. We demonstrate our method on a vision-based aircraft collision avoidance problem and show improvements in terms of both accuracy and efficiency over the Gaussian process and threshold bandit baselines.