mlsanityData to 14 Oct 2026

Constrained Bayesian Optimization with Noisy Experiments

arXiv 1706.07094 ↗First posted 21 Jun 2017stat.ML cs.LG stat.APcited by 104

Not on the map. Why?

Abstract (arXiv)

Randomized experiments are the gold standard for evaluating the effects of changes to real-world systems. Data in these tests may be difficult to collect and outcomes may have high variance, resulting in potentially large measurement error. Bayesian optimization is a promising technique for efficiently optimizing multiple continuous parameters, but existing approaches degrade in performance when the noise level is high, limiting its applicability to many randomized experiments. We derive an expression for expected improvement under greedy batch optimization with noisy observations and noisy constraints, and develop a quasi-Monte Carlo approximation that allows it to be efficiently optimized. Simulations with synthetic functions show that optimization performance on noisy, constrained problems outperforms existing methods. We further demonstrate the effectiveness of the method with two real-world experiments conducted at Facebook: optimizing a ranking system, and optimizing server compiler flags.

Key facts. Cited by 104 papers in the map (data through 14 Oct 2026). First posted on arXiv 21 Jun 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 21; 2022: 20; 2023: 21; 2024: 15; 2025: 12; 2026: 15.

104 papers.

Sort:

By area:

  1. 2023Unexpected Improvements to Expected Improvement for Bayesian Optimizationcited by 149
  2. 2021Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020cited by 148
  3. 2021High-Dimensional Bayesian Optimization with Sparse Axis-Aligned Subspacescited by 141
  4. 2021Parallel Bayesian Optimization of Multiple Noisy Objectives with Expected Hypervolume Improvementcited by 118
  5. 2022Accelerating Bayesian Optimization for Biological Sequence Design with Denoising Autoencoderscited by 102
  6. 2022Recent Advances in Bayesian Optimizationcited by 76
  7. 2023PFNs4BO: In-Context Learning for Bayesian Optimizationcited by 73
  8. 2022Preference Exploration for Efficient Bayesian Optimization with Multiple Outcomescited by 42
  9. 2022Robust Multi-Objective Bayesian Optimization Under Input Noisecited by 37
  10. 2022Discovering Many Diverse Solutions with Bayesian Optimizationcited by 31
  11. 2022Bayesian Optimization with Conformal Prediction Setscited by 28
  12. 2021Sequential- and Parallel- Constrained Max-value Entropy Search via Information Lower Boundcited by 23
38 more
  1. 2021Conditioning Sparse Variational Gaussian Processes for Online Decision-makingcited by 23
  2. 2023Self-Correcting Bayesian Optimization through Bayesian Active Learningcited by 22
  3. 2023A General Framework for User-Guided Bayesian Optimizationcited by 22
  4. 2021Kernel Interpolation for Scalable Online Gaussian Processescited by 18
  5. 2021Apollo: Transferable Architecture Explorationcited by 17
  6. 2023Equal Opportunity of Coverage in Fair Regressioncited by 16
  7. 2022Scalable Bayesian Optimization Using Vecchia Approximations of Gaussian Processescited by 11
  8. 2024Expensive Multi-Objective Bayesian Optimization Based on Diffusion Modelscited by 10
  9. 2021Bayesian Optimisation for Constrained Problemscited by 9
  10. 2021Looper: An end-to-end ML platform for product decisionscited by 9
  11. 2021A portfolio approach to massively parallel Bayesian optimizationcited by 9
  12. 2022c-TPE: Tree-structured Parzen Estimator with Inequality Constraints for Expensive Hyperparameter Optimizationcited by 9
  13. 2021Non-smooth Bayesian Optimization in Tuning Problemscited by 8
  14. 2022PropertyDAG: Multi-objective Bayesian optimization of partially ordered, mixed-variable properties for biological sequence designcited by 8
  15. 2023Constrained Bayesian Optimization Under Partial Observations: Balanced Improvements and Provable Convergencecited by 8
  16. 2022On Noisy Evaluation in Federated Hyperparameter Tuningcited by 7
  17. 2024Physics-Informed Bayesian Optimization of Variational Quantum Circuitscited by 7
  18. 2022Sparse Bayesian Optimizationcited by 6
  19. 2022Adjusted Expected Improvement for Cumulative Regret Minimization in Noisy Bayesian Optimizationcited by 6
  20. 2023Constrained Causal Bayesian Optimizationcited by 6
  21. 2023Adaptive Batch Sizes for Active Learning A Probabilistic Numerics Approachcited by 6
  22. 2024Constrained Multi-objective Bayesian Optimization through Optimistic Constraints Estimationcited by 6
  23. 2021Efficient Exploration in Binary and Preferential Bayesian Optimizationcited by 5
  24. 2021Failure-averse Active Learning for Physics-constrained Systemscited by 5
  25. 2023High-dimensional Bayesian Optimization with Group Testingcited by 5
  26. 2023A Corrected Expected Improvement Acquisition Function Under Noisy Observationscited by 5
  27. 2023Joint Composite Latent Space Bayesian Optimizationcited by 5
  28. 2023Risk-Controlling Model Selection via Guided Bayesian Optimizationcited by 5
  29. 2026BayeSQP: Bayesian Optimization through Sequential Quadratic Programmingcited by 5
  30. 2021Two-step Lookahead Bayesian Optimization with Inequality Constraintscited by 4
  31. 2023Quantum approximate optimization via learning-based adaptive optimizationcited by 4
  32. 2023Black-Box Optimization with Implicit Constraints for Public Policycited by 4
  33. 2024TS-RSR: A provably efficient approach for batch Bayesian Optimizationcited by 4
  34. 2024Active learning for affinity prediction of antibodiescited by 4
  35. 2024Sample-efficient Bayesian Optimisation Using Known Invariancescited by 4
  36. 2025Regret Analysis of Posterior Sampling-Based Expected Improvement for Bayesian Optimizationcited by 4
  37. 2022Partial Likelihood Thompson Samplingcited by 3
  38. 2022Mind the Gap: Measuring Generalization Performance Across Multiple Objectivescited by 3

50 most cited of 104 All 104 (JSON)