mlsanityData to 7 Oct 2026

Converge Then Diversify: Decoupling Convergence and Diversity in Multi-Objective Bayesian Optimisation

arXiv 2609.13396 ↗First posted 11 Sep 2026cs.AI stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

Multi-objective Bayesian optimisation (MOBO) is a sample-efficient approach for optimising expensive black-box functions with multiple objectives. In MOBO, the goal is to adequately approximate the Pareto front; that is, to obtain a high-quality solution set with 1) good convergence (closeness to the Pareto front) and 2) good diversity (spread across the Pareto front). Existing MOBO methods typically aim to accomplish these two tasks simultaneously, i.e., driving the search towards the Pareto front while maintaining a diverse set of nondominated solutions, such that the solutions, ideally, can gradually approach the entire front. When sufficient search budgets are available, this approach is effective. However, considering both convergence and diversity throughout the search is not easy and requires careful design. Under very tight budgets, there may not be enough solutions generated to be able to simultaneously approach the entire Pareto front. To address this issue, this paper proposes a textitconverge-then-diversify (CTD) approach that decouples convergence and diversity into two stages. In the first stage, CTD focuses on convergence, aiming to quickly drive the search toward a single point on the Pareto front. In the second stage, CTD focuses on diversity, aiming to spread solutions across the front. We present two simple instantiations of CTD by using widely adopted acquisition functions in the area. Experimental results show that, across all 446 pairwise comparisons, CTD statistically outperforms state-of-the-art methods in 72.9% of the cases, performs equivalently in 21.1%, and is statistically worse in only 6.1%, with the advantage being particularly evident in settings with very tight evaluation budgets or in high-dimensional problems.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 89 of its 155 references that are in the map. First posted on arXiv 11 Sep 2026.

Builds on

89 of 155 references.

References · 89

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2018GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Accelerationcited by 603
  3. 2019Scalable Global Optimization via Local Bayesian Optimizationcited by 359
  4. 2013Bayesian Optimization in a Billion Dimensions via Random Embeddingscited by 178
  5. 2020Differentiable Expected Hypervolume Improvement for Parallel Multi-Objective Bayesian Optimizationcited by 176
  6. 2023Unexpected Improvements to Expected Improvement for Bayesian Optimizationcited by 144
  7. 2021High-Dimensional Bayesian Optimization with Sparse Axis-Aligned Subspacescited by 139
  8. 2017Multi-fidelity Bayesian Optimisation with Continuous Approximationscited by 120
  9. 2017Batched Large-scale Bayesian Optimization in High-dimensional Spacescited by 116
  10. 2021Parallel Bayesian Optimization of Multiple Noisy Objectives with Expected Hypervolume Improvementcited by 115
  11. 2014Bayesian Optimization with Unknown Constraintscited by 109
  12. 2017Constrained Bayesian Optimization with Noisy Experimentscited by 103
38 more
  1. 2018A Flexible Framework for Multi-Objective Bayesian Optimization using Random Scalarizationscited by 98
  2. 2019Practical Multi-fidelity Bayesian Optimization for Hyperparameter Tuningcited by 91
  3. 2020Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimizationcited by 89
  4. 2015Predictive Entropy Search for Multi-objective Bayesian Optimizationcited by 88
  5. 2021Multi-Objective Bayesian Optimization over High-Dimensional Search Spacescited by 87
  6. 2024Vanilla Bayesian Optimization Performs Great in High Dimensionscited by 87
  7. 2020Max-value Entropy Search for Multi-Objective Bayesian Optimization with Constraintscited by 81
  8. 2021A survey on high-dimensional Gaussian process modeling with application to Bayesian optimizationcited by 76
  9. 2022Recent Advances in Bayesian Optimizationcited by 74
  10. 2020An Easy-to-use Real-world Multi-objective Optimization Problem Suitecited by 71
  11. 2020Random Hypervolume Scalarizations for Provable Multi-Objective Black Box Optimizationcited by 60
  12. 2019Multi-objective Bayesian Optimization using Pareto-frontier Entropycited by 57
  13. 2018A General Framework for Multi-fidelity Bayesian Optimization with Gaussian Processescited by 52
  14. 2022Pareto Set Learning for Expensive Multi-Objective Optimizationcited by 51
  15. 2024Smooth Tchebycheff Scalarization for Multi-Objective Optimizationcited by 50
  16. 2022Constrained Efficient Global Optimization of Expensive Black-box Functionscited by 49
  17. 2023Sample-efficient Multi-objective Molecular Optimization with GFlowNetscited by 48
  18. 2021GIBBON: General-purpose Information-Based Bayesian OptimisatioNcited by 47
  19. 2020Multi-Fidelity Bayesian Optimization via Deep Neural Networkscited by 46
  20. 2022Joint Entropy Search for Multi-objective Bayesian Optimizationcited by 45
  21. 2022Uncertainty-Aware Search Framework for Multi-Objective Bayesian Optimizationcited by 43
  22. 2022Multi-Objective Hyperparameter Optimization in Machine Learning -- An Overviewcited by 37
  23. 2022Robust Multi-Objective Bayesian Optimization Under Input Noisecited by 36
  24. 2023Are Random Decompositions all we need in High Dimensional Bayesian Optimisation?cited by 35
  25. 2024Standard Gaussian Process is All You Need for High-Dimensional Bayesian Optimizationcited by 33
  26. 2016Bayesian optimization under mixed constraints with a slack-variable augmented Lagrangiancited by 29
  27. 2021TREGO: a Trust-Region Framework for Efficient Global Optimizationcited by 28
  28. 2016Predictive Entropy Search for Multi-objective Bayesian Optimization with Constraintscited by 27
  29. 2023Bounce: Reliable High-Dimensional Bayesian Optimization for Combinatorial and Mixed Spacescited by 27
  30. 2019Efficient Computation of Expected Hypervolume Improvement Using Box Decomposition Algorithmscited by 24
  31. 2020Multi-Fidelity Multi-Objective Bayesian Optimization: An Output Space Entropy Search Approachcited by 22
  32. 2020High-Dimensional Bayesian Optimization via Tree-Structured Additive Modelscited by 22
  33. 2024A survey and benchmark of high-dimensional Bayesian optimization of discrete sequencescited by 22
  34. 2021Automatic Termination for Hyperparameter Optimizationcited by 21
  35. 2019Randomly Projected Additive Gaussian Processes for Regressioncited by 19
  36. 2025Understanding High-Dimensional Bayesian Optimizationcited by 19
  37. 2023Comparison of High-Dimensional Bayesian Optimization Algorithms on BBOBcited by 18
  38. 2019Scalarizing Functions in Bayesian Multiobjective Optimizationcited by 15

50 most cited of 89 All 89 (JSON)

Built on by

Not cited yet.