mlsanityData to 7 Oct 2026

Sparse Bayesian Optimization

arXiv 2203.01900 ↗First posted 3 Mar 2022cs.LG cs.AI math.OCcited by 6

Not on the map. Why?

Abstract (arXiv)

Bayesian optimization (BO) is a powerful approach to sample-efficient optimization of black-box objective functions. However, the application of BO to areas such as recommendation systems often requires taking the interpretability and simplicity of the configurations into consideration, a setting that has not been previously studied in the BO literature. To make BO useful for this setting, we present several regularization-based approaches that allow us to discover sparse and more interpretable configurations. We propose a novel differentiable relaxation based on homotopy continuation that makes it possible to target sparsity by working directly with L0 regularization. We identify failure modes for regularized BO and develop a hyperparameter-free method, sparsity exploring Bayesian optimization (SEBO) that seeks to simultaneously maximize a target objective and sparsity. SEBO and methods based on fixed regularization are evaluated on synthetic and real-world problems, and we show that we are able to efficiently optimize for sparsity.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 19 of its 47 references that are in the map. First posted on arXiv 3 Mar 2022.

Builds on

19 of 47 references.

References · 19

  1. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  2. 2016Automatic chemical design using a data-driven continuous representation of moleculescited by 762
  3. 2021Interpretable Machine Learning: Fundamental Principles and 10 Grand Challengescited by 364
  4. 2019Scalable Global Optimization via Local Bayesian Optimizationcited by 359
  5. 2017Max-value Entropy Search for Efficient Bayesian Optimizationcited by 202
  6. 2014Predictive Entropy Search for Efficient Global Optimization of Black-box Functionscited by 201
  7. 2013Bayesian Optimization in a Billion Dimensions via Random Embeddingscited by 178
  8. 2020Differentiable Expected Hypervolume Improvement for Parallel Multi-Objective Bayesian Optimizationcited by 176
  9. 2021Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020cited by 146
  10. 2021High-Dimensional Bayesian Optimization with Sparse Axis-Aligned Subspacescited by 139
  11. 2021Parallel Bayesian Optimization of Multiple Noisy Objectives with Expected Hypervolume Improvementcited by 115
  12. 2018Maximizing acquisition functions for Bayesian optimizationcited by 113
7 more
  1. 2019Optimal Sparse Decision Treescited by 104
  2. 2015Batch Bayesian Optimization via Local Penalizationcited by 103
  3. 2017Constrained Bayesian Optimization with Noisy Experimentscited by 103
  4. 2019Combinatorial Bayesian Optimization using the Graph Cartesian Productcited by 78
  5. 2015Supersparse Linear Integer Models for Optimized Medical Scoring Systemscited by 76
  6. 2015Unbounded Bayesian Optimization via Regularizationcited by 13
  7. 2020Variable selection for Gaussian process regression through a sparse projectioncited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 3; 2024: 0; 2025: 0; 2026: 2.

6 papers.

  1. 2022Bayesian Optimization over Discrete and Mixed Spaces via Probabilistic Reparameterizationcited by 35
  2. 2023Looping in the Human Collaborative and Explainable Bayesian Optimizationcited by 27
  3. 2023Network Cascade Vulnerability using Constrained Bayesian Optimizationcited by 0
  4. 2023Learning Relevant Contextual Variables Within Bayesian Optimizationcited by 0
  5. 2026Dynamic Hyperparameter Importance for Efficient Multi-Objective Optimizationcited by 0
  6. 2026BONSAI: Bayesian Optimization with Natural Simplicity and Interpretabilitycited by 0