mlsanityData to 14 Oct 2026

Efficient Conditioning Why Pseudo Observation Batch Bayesian Optimization Works When It Does not

arXiv 2605.18819 ↗First posted 12 May 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Constant Liar (CL), Kriging Believer (KB), and fantasy models are widely used for batch selection in parallel Bayesian Optimization, yet a unified theory explaining their effectiveness and conditions under which they fail has been lacking. We identify efficient conditioning as the key surrogate property the ability to update predictions in closed form when data is augmented. We prove that Gaussian Processes satisfy this requirement, producing provably distinct batch points with separation of order l, and that this holds for any acquisition function monotonically non decreasing in posterior uncertainty (EI, UCB, PI), with qualitatively similar behavior for Thompson Sampling. We unify CL, KB, and fantasy models as instances of a single conditioning mechanism differing only in the lie value distribution, and draw quantitative connections to Local Penalization (LP) and qualitative connections to Determinantal Point Processes (DPPs). To disentangle model structure from optimizer randomness, we introduce the Structural Diversity Diagnostic (SDD), a reusable methodology for testing surrogate compatibility. Experiments on Hartmann6D, Ackley 8D, Levy10D, and SVM hyperparameter tuning validate all theoretical predictions: CL or KBs implicit penalty matches or outperforms explicit LP greedy conditioning achieves convergence on par with joint qEI efficient conditioning extends to Multiquadric RBF networks; and parametric surrogates produce degenerate batches even when fully retrained (random forests), while neural networks regain diversity only at 15x the wall clock cost of GP conditioning. Robustness is confirmed across multiple initial datasets and under observation noise.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 8 of its 26 references that are in the map. First posted on arXiv 12 May 2026.

Builds on

8 of 26 references.

References · 8

  1. 2013Gaussian Processes for Big Datacited by 383
  2. 2015Scalable Bayesian Optimization Using Deep Neural Networkscited by 284
  3. 2017Batched Large-scale Bayesian Optimization in High-dimensional Spacescited by 117
  4. 2018Maximizing acquisition functions for Bayesian optimizationcited by 114
  5. 2015Batch Bayesian Optimization via Local Penalizationcited by 104
  6. 2013Parallel Gaussian Process Optimization with Upper Confidence Bound and Pure Explorationcited by 50
  7. 2016Batched Gaussian Process Bandit Optimization via Determinantal Point Processescited by 33
  8. 2019Asynchronous Batch Bayesian Optimisation with Improved Local Penalisationcited by 17

Built on by

Not cited yet.