Scalable Bayesian optimization with high-dimensional outputs using randomized prior networks
Not on the map. Why?
Abstract (arXiv)
Several fundamental problems in science and engineering consist of global optimization tasks involving unknown high-dimensional (black-box) functions that map a set of controllable variables to the outcomes of an expensive experiment. Bayesian Optimization (BO) techniques are known to be effective in tackling global optimization problems using a relatively small number objective function evaluations, but their performance suffers when dealing with high-dimensional outputs. To overcome the major challenge of dimensionality, here we propose a deep learning framework for BO and sequential decision making based on bootstrapped ensembles of neural architectures with randomized priors. Using appropriate architecture choices, we show that the proposed framework can approximate functional relationships between design variables and quantities of interest, even in cases where the latter take values in high-dimensional vector spaces or even infinite-dimensional function spaces. In the context of BO, we augmented the proposed probabilistic surrogates with re-parameterized Monte Carlo approximations of multiple-point (parallel) acquisition functions, as well as methodological extensions for accommodating black-box constraints and multi-fidelity information sources. We test the proposed framework against state-of-the-art methods for BO and demonstrate superior performance across several challenging tasks with high-dimensional outputs, including a constrained multi-fidelity optimization task involving shape optimization of rotor blades in turbo-machinery.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 43 of its 107 references that are in the map. First posted on arXiv 14 Feb 2023.
Builds on
43 of 107 references.
References · 43
- 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
- 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 1,965
- 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,194
- 2015An Introduction to Convolutional Neural Networkscited by 510
- 2019Deep Ensembles: A Loss Landscape Perspectivecited by 438
- 2017A Tutorial on Thompson Samplingcited by 420
- 2018Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Networkcited by 415
- 2019Scalable Global Optimization via Local Bayesian Optimizationcited by 359
- 2018Randomized Prior Functions for Deep Reinforcement Learningcited by 218
- 2020Differentiable Expected Hypervolume Improvement for Parallel Multi-Objective Bayesian Optimizationcited by 176
- 2021Active Learning by Acquiring Contrastive Examplescited by 154
- 2015High Dimensional Bayesian Optimisation and Bandits via Additive Modelscited by 137
31 more
- 2018Maximizing acquisition functions for Bayesian optimizationcited by 113
- 2020Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimizationcited by 89
- 2020Scalable Constrained Bayesian Optimizationcited by 84
- 2016Multi-Information Source Optimizationcited by 83
- 2016The Parallel Knowledge Gradient Method for Batch Bayesian Optimizationcited by 73
- 2017Bayesian Optimization with Gradientscited by 59
- 2017Batched High-dimensional Bayesian Optimization via Structural Kernel Learningcited by 51
- 2013Parallel Gaussian Process Optimization with Upper Confidence Bound and Pure Explorationcited by 50
- 2019Bayesian Optimization of Composite Functionscited by 45
- 2020Bayesian Neural Ordinary Differential Equationscited by 44
- 2020Efficient Nonmyopic Bayesian Optimization via One-Shot Multi-Step Treescited by 41
- 2022Scalable Uncertainty Quantification for Deep Operator Networks using Randomized Priorscited by 39
- 2019BINOCULARS for Efficient, Nonmyopic Sequential Experimental Designcited by 35
- 2020Bayesian Optimization of Risk Measurescited by 35
- 2021Bayesian Optimization with High-Dimensional Outputscited by 32
- 2022Thinking inside the box: A tutorial on grey-box Bayesian optimizationcited by 26
- 2020On spike-and-slab priors for Bayesian equation discovery of nonlinear dynamical systems via sparse linear regressioncited by 24
- 2020Optimizing Coverage and Capacity in Cellular Networks using Machine Learningcited by 21
- 2020Efficient Rollout Strategies for Bayesian Optimizationcited by 19
- 2020No-regret Algorithms for Multi-task Bayesian Optimizationcited by 18
- 2018Fast Gaussian Process Based Gradient Matching for Parameter Identification in Systems of Nonlinear ODEscited by 17
- 2020Bayesian differential programming for robust systems identification under uncertaintycited by 16
- 2021Gaussian processes meet NeuralODEs: A Bayesian framework for learning the dynamics of partially observed systems from scarce and noisy datacited by 13
- 2021An Efficient Batch Constrained Bayesian Optimization Approach for Analog Circuit Synthesis via Multi-objective Acquisition Ensemblecited by 13
- 2020Interferobot: aligning an optical interferometer by a reinforcement learning agentcited by 12
- 2019Multi-fidelity classification using Gaussian processes: accelerating the prediction of large-scale computational modelscited by 8
- 2021Optimizing High-Dimensional Physics Simulations via Composite Bayesian Optimizationcited by 6
- 2018On a New Improvement-Based Acquisition Function for Bayesian Optimizationcited by 5
- 2020Informative Path Planning for Extreme Anomaly Detection in Environment Exploration and Monitoringcited by 5
- 2020Output-Weighted Optimal Sampling for Bayesian Experimental Design and Uncertainty Quantificationcited by 2
- 2019High-dimensional Bayesian optimization using low-dimensional feature spacescited by 1
Built on by
Citing papers per year: 2023: 2.
2 papers.