mlsanityData to 14 Oct 2026

A Bayesian Optimization Framework for Finding Local Optima in Expensive Multi-Modal Functions

arXiv 2210.06635 ↗First posted 13 Oct 2022math.OC cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

Bayesian optimization (BO) is a popular global optimization scheme for sample-efficient optimization in domains with expensive function evaluations. The existing BO techniques are capable of finding a single global optimum solution. However, finding a set of global and local optimum solutions is crucial in a wide range of real-world problems, as implementing some of the optimal solutions might not be feasible due to various practical restrictions (e.g., resource limitation, physical constraints, etc.). In such domains, if multiple solutions are known, the implementation can be quickly switched to another solution, and the best possible system performance can still be obtained. This paper develops a multimodal BO framework to effectively find a set of local/global solutions for expensive-to-evaluate multimodal objective functions. We consider the standard BO setting with Gaussian process regression representing the objective function. We analytically derive the joint distribution of the objective function and its first-order derivatives. This joint distribution is used in the body of the BO acquisition functions to search for local optima during the optimization process. We introduce variants of the well-known BO acquisition functions to the multimodal setting and demonstrate the performance of the proposed framework in locating a set of local optimum solutions using multiple optimization problems.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 22 of its 67 references that are in the map. First posted on arXiv 13 Oct 2022.

Builds on

22 of 67 references.

References · 22

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,546
  3. 2018A Tutorial on Bayesian Optimizationcited by 744
  4. 2019Scalable Global Optimization via Local Bayesian Optimizationcited by 374
  5. 2015Gradient-based Hyperparameter Optimization through Reversible Learningcited by 360
  6. 2014Predictive Entropy Search for Efficient Global Optimization of Black-box Functionscited by 204
  7. 2015Bayesian optimization for materials designcited by 80
  8. 2014A Stochastic Quasi-Newton Method for Large-Scale Optimizationcited by 69
  9. 2019Bayesian Optimization for Materials Design with Mixed Quantitative and Qualitative Variablescited by 66
  10. 2017Bayesian Optimization with Gradientscited by 59
  11. 2015Predictive Entropy Search for Bayesian Optimization with Unknown Constraintscited by 51
  12. 2020Efficient Nonmyopic Bayesian Optimization via One-Shot Multi-Step Treescited by 41
10 more
  1. 2015GLASSES: Relieving The Myopia Of Bayesian Optimisationcited by 38
  2. 2021Local policy search with Bayesian optimizationcited by 38
  3. 2019BINOCULARS for Efficient, Nonmyopic Sequential Experimental Designcited by 35
  4. 2017Virtual vs. Real: Trading Off Simulations and Physical Experiments in Reinforcement Learning with Bayesian Optimizationcited by 34
  5. 2016Bayesian optimization under mixed constraints with a slack-variable augmented Lagrangiancited by 29
  6. 2020Network-Aware Optimization of Distributed Learning for Fog Computingcited by 27
  7. 2019Constrained Bayesian Optimization with Max-Value Entropy Searchcited by 20
  8. 2017Practical Bayesian Optimization for Model Fitting with Bayesian Adaptive Direct Searchcited by 11
  9. 2018Hierarchical ResNeXt Models for Breast Cancer Histology Image Classificationcited by 4
  10. 2021Two-step Lookahead Bayesian Optimization with Inequality Constraintscited by 4

Built on by

Cited by year

Citing papers per year: 2023: 5; 2024: 5; 2025: 2.

12 papers.

  1. 2024AdaZeta: Adaptive Zeroth-Order Tensor-Train Adaption for Memory-Efficient Large Language Models Fine-Tuningcited by 25
  2. 2023Every Parameter Matters: Ensuring the Convergence of Federated Learning with Dynamic Heterogeneous Models Reductioncited by 24
  3. 2023MAC-PO: Multi-Agent Experience Replay via Collective Priority Optimizationcited by 20
  4. 2024Collaborative AI Teaming in Unknown Environments via Active Goal Deductioncited by 6
  5. 2024Cooperative Backdoor Attack in Decentralized Reinforcement Learning with Theoretical Guaranteecited by 6
  6. 2024Bayesian Optimization through Gaussian Cox Process Models for Spatio-temporal Datacited by 5
  7. 2023Exploiting Partial Common Information Microstructure for Multi-Modal Brain Tumor Segmentationcited by 3
  8. 2023ReMIX: Regret Minimization for Monotonic Value Function Factorization in Multiagent Reinforcement Learningcited by 3
  9. 2024RGMDT: Return-Gap-Minimizing Decision Tree Extraction in Non-Euclidean Metric Spacecited by 2
  10. 2023Statistically Efficient Variance Reduction with Double Policy Estimation for Off-Policy Evaluation in Sequence-Modeled Reinforcement Learningcited by 0
  11. 2025A Balanced Approach of Rapid Genetic Exploration and Surrogate Exploitation for Hyperparameter Optimizationcited by 0
  12. 2025Global Optimization on Graph-Structured Data via Gaussian Processes with Spectral Representationscited by 0