# Active learning for distributionally robust level-set estimation

arXiv: https://arxiv.org/abs/2102.04000 · HTML: https://mlsanity.com/paper/2102.04000/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 34 references that are in the map.
- First posted on arXiv 8 Feb 2021.

## Builds on (most cited first, 15 of 15)

- [Stagewise Safe Bayesian Optimization with Gaussian Processes](https://mlsanity.com/paper/1806.07555/) (2018)
- [Safe Exploration in Finite Markov Decision Processes with Gaussian Processes](https://mlsanity.com/paper/1606.04753/) (2016)
- [Adversarially Robust Optimization with Gaussian Processes](https://mlsanity.com/paper/1810.10775/) (2018)
- [Distributionally Robust Bayesian Optimization](https://mlsanity.com/paper/2002.09038/) (2020)
- [Bayesian Optimization with Expensive Integrands](https://mlsanity.com/paper/1803.08661/) (2018)
- [Noisy-Input Entropy Search for Efficient Robust Bayesian Optimization](https://mlsanity.com/paper/2002.02820/) (2020)
- [Bayesian optimisation under uncertain inputs](https://mlsanity.com/paper/1902.07908/) (2019)
- [Mean-Variance Analysis in Bayesian Optimization under Uncertainty](https://mlsanity.com/paper/2009.08166/) (2020)
- [Active Multi-Information Source Bayesian Quadrature](https://mlsanity.com/paper/1903.11331/) (2019)
- [Robust Super-Level Set Estimation using Gaussian Processes](https://mlsanity.com/paper/1811.09977/) (2018)
- [Distributionally Robust Bayesian Quadrature Optimization](https://mlsanity.com/paper/2001.06814/) (2020)
- [Bayesian Experimental Design for Finding Reliable Level Set under Input Uncertainty](https://mlsanity.com/paper/1910.12043/) (2019)
- [Active learning for level set estimation under input uncertainty and its extensions](https://mlsanity.com/paper/1909.06064/) (2019)
- [Bayesian Quadrature Optimization for Probability Threshold Robustness Measure](https://mlsanity.com/paper/2006.11986/) (2020)
- [Active learning for enumerating local minima based on Gaussian process derivatives](https://mlsanity.com/paper/1903.03279/) (2019)

## Built on by (most cited first, 10 of 10)

- [Bayesian Optimization for Cascade-type Multi-stage Processes](https://mlsanity.com/paper/2111.08330/) (2021)
- [Bayesian Optimization for Distributionally Robust Chance-constrained Problem](https://mlsanity.com/paper/2201.13112/) (2022)
- [Bounding Box-based Multi-objective Bayesian Optimization of Risk Measures under Input Uncertainty](https://mlsanity.com/paper/2301.11588/) (2023)
- [Active Level Set Estimation for Continuous Search Space with Theoretical Guarantee](https://mlsanity.com/paper/2402.16237/) (2024)
- [Active Learning for Level Set Estimation Using Randomized Straddle Algorithms](https://mlsanity.com/paper/2408.03144/) (2024)
- [Distributionally Robust Active Learning for Gaussian Process Regression](https://mlsanity.com/paper/2502.16870/) (2025)
- [An Active Learning Reliability Method for Systems with Partially Defined Performance Functions](https://mlsanity.com/paper/2210.02168/) (2022)
- [Efficient Determination of Safety Requirements for Perception Systems](https://mlsanity.com/paper/2307.01371/) (2023)
- [Attacking Motion Planners Using Adversarial Perception Errors](https://mlsanity.com/paper/2311.12722/) (2023)
- [An (ε,δ)-accurate level set estimation with a stopping criterion](https://mlsanity.com/paper/2503.20272/) (2025)

## Abstract (arXiv)

Many cases exist in which a black-box function f with high evaluation cost depends on two types of variables bm x and bm w, where bm x is a controllable design variable and bm w are uncontrollable environmental variables that have random variation following a certain distribution P. In such cases, an important task is to find the range of design variables bm x such that the function f(bm x, bm w) has the desired properties by incorporating the random variation of the environmental variables bm w. A natural measure of robustness is the probability that f(bm x, bm w) exceeds a given threshold h, which is known as the probability threshold robustness (PTR) measure in the literature on robust optimization. However, this robustness measure cannot be correctly evaluated when the distribution P is unknown. In this study, we addressed this problem by considering the textitdistributionally robust PTR (DRPTR) measure, which considers the worst-case PTR within given candidate distributions. Specifically, we studied the problem of efficiently identifying a reliable set H, which is defined as a region in which the DRPTR measure exceeds a certain desired probability α, which can be interpreted as a level set estimation (LSE) problem for DRPTR. We propose a theoretically grounded and computationally efficient active learning method for this problem. We show that the proposed method has theoretical guarantees on convergence and accuracy, and confirmed through numerical experiments that the proposed method outperforms existing methods.