mlsanityData to 14 Oct 2026

Bayesian Optimization with Conformal Prediction Sets

arXiv 2210.12496 ↗First posted 22 Oct 2022cs.LG stat.MLcited by 28

Not on the map. Why?

Abstract (arXiv)

Bayesian optimization is a coherent, ubiquitous approach to decision-making under uncertainty, with applications including multi-arm bandits, active learning, and black-box optimization. Bayesian optimization selects decisions (i.e. objective function queries) with maximal expected utility with respect to the posterior distribution of a Bayesian model, which quantifies reducible, epistemic uncertainty about query outcomes. In practice, subjectively implausible outcomes can occur regularly for two reasons: 1) model misspecification and 2) covariate shift. Conformal prediction is an uncertainty quantification method with coverage guarantees even for misspecified models and a simple mechanism to correct for covariate shift. We propose conformal Bayesian optimization, which directs queries towards regions of search space where the model predictions have guaranteed validity, and investigate its behavior on a suite of black-box optimization tasks and tabular ranking tasks. In many cases we find that query coverage can be significantly improved without harming sample-efficiency.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Builds on 32 of its 69 references that are in the map. First posted on arXiv 22 Oct 2022.

Builds on

32 of 69 references.

References · 32

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 792
  3. 2016Automatic chemical design using a data-driven continuous representation of moleculescited by 765
  4. 2018A Tutorial on Bayesian Optimizationcited by 744
  5. 2018GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Accelerationcited by 609
  6. 2019Conformal Prediction Under Covariate Shiftcited by 519
  7. 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 482
  8. 2019Scalable Global Optimization via Local Bayesian Optimizationcited by 374
  9. 2021Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Developmentcited by 264
  10. 2020Differentiable Expected Hypervolume Improvement for Parallel Multi-Objective Bayesian Optimizationcited by 179
  11. 2021Testing for Outliers with Conformal p-valuescited by 134
  12. 2019Learning with Good Feature Representations in Bandits and in RL with a Generative Modelcited by 127
20 more
  1. 2021Parallel Bayesian Optimization of Multiple Noisy Objectives with Expected Hypervolume Improvementcited by 118
  2. 2021Learning Optimal Conformal Classifierscited by 112
  3. 2017Constrained Bayesian Optimization with Noisy Experimentscited by 104
  4. 2022Accelerating Bayesian Optimization for Biological Sequence Design with Denoising Autoencoderscited by 102
  5. 2021Multi-Objective Bayesian Optimization over High-Dimensional Search Spacescited by 87
  6. 2021Adapting to Misspecification in Contextual Banditscited by 81
  7. 2017The reparameterization trick for acquisition functionscited by 63
  8. 2021Misspecified Gaussian Process Bandit Optimizationcited by 55
  9. 2021Conformal Bayesian Computationcited by 39
  10. 2022Robust Multi-Objective Bayesian Optimization Under Input Noisecited by 37
  11. 2018Regret bounds for meta Bayesian optimization with an unknown Gaussian process priorcited by 31
  12. 2022Efficient and Differentiable Conformal Prediction with General Function Classescited by 26
  13. 2021Conditioning Sparse Variational Gaussian Processes for Online Decision-makingcited by 23
  14. 2021Bayes-optimal prediction with frequentist coverage controlcited by 20
  15. 2021Stable Conformal Prediction Setscited by 19
  16. 2021Kernel Interpolation for Scalable Online Gaussian Processescited by 18
  17. 2021Risk-averse Heteroscedastic Bayesian Optimizationcited by 17
  18. 2022Modular Conformal Calibrationcited by 15
  19. 2019Achieving Robustness to Aleatoric Uncertainty with Heteroscedastic Bayesian Optimisationcited by 11
  20. 2014Heteroscedastic Treed Bayesian Optimisationcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 8; 2024: 9; 2025: 5; 2026: 6.

28 papers.

Sort:

  1. 2024Conformal Validity Guarantees Exist for Any Data Distribution (and How to Find Them)cited by 26
  2. 2023Self-Correcting Bayesian Optimization through Bayesian Active Learningcited by 22
  3. 2024Label-wise Aleatoric and Epistemic Uncertainty Quantificationcited by 22
  4. 2023Second-Order Uncertainty Quantification: Variance-Based Measurescited by 19
  5. 2025WATCH: Adaptive Monitoring for AI Deployments via Weighted-Conformal Martingalescited by 16
  6. 2023Bayesian Optimization with Formal Safety Guarantees via Online Conformal Predictioncited by 10
  7. 2024Conformalized Interval Arithmetic with Symmetric Calibrationcited by 8
  8. 2024Conformal Approach To Gaussian Process Surrogate Evaluation With Coverage Guaranteescited by 7
  9. 2025Conformal Prediction as Bayesian Quadraturecited by 6
  10. 2026Conformal Policy Controlcited by 6
  11. 2023Conformalized Deep Splines for Optimal and Efficient Prediction Setscited by 5
  12. 2023Risk-Controlling Model Selection via Guided Bayesian Optimizationcited by 5
16 more
  1. 2024CUQDS: Conformal Uncertainty Quantification under Distribution Shift for Trajectory Predictioncited by 5
  2. 2023Beyond the training set: an intuitive method for detecting distribution shift in model-based optimizationcited by 4
  3. 2024Efficient Online Set-valued Classification with Bandit Feedbackcited by 3
  4. 2024Online scalable Gaussian processes with conformal prediction for guaranteed coveragecited by 3
  5. 2023Is novelty predictable?cited by 2
  6. 2025Smart Surrogate Losses for Contextual Stochastic Linear Optimization with Robust Constraintscited by 2
  7. 2024Anchor-Based Heteroscedastic Noise for Preferential Bayesian Optimizationcited by 1
  8. 2024Robust Bayesian Optimization via Localized Online Conformal Predictioncited by 1
  9. 2023On the Out-of-Distribution Coverage of Combining Split Conformal Prediction and Bayesian Deep Learningcited by 0
  10. 2025Aerial Image Classification in Scarce and Unconstrained Environments via Conformal Predictioncited by 0
  11. 2025Conformal bandits: bringing statistical validity and reward efficiency under weak arm separabilitycited by 0
  12. 2026Online Sharp-Calibrated Bayesian Optimizationcited by 0
  13. 2026Goal-Oriented Lower-Tail Calibration of Gaussian Processes for Bayesian Optimizationcited by 0
  14. 2026AK-MCS-C2 : Active Kriging Monte Carlo Simulation method with conformal certification for failure probability estimationcited by 0
  15. 2026Adapting Nonstationary Multi-output Gaussian Processes to Bayesian Optimizationcited by 0
  16. 2026Coverage You Can Steer: Online Conformal Calibration for RL-Driven Hardware-Aware NAScited by 0