mlsanityData to 7 Oct 2026

Raiders of the Lost Architecture: Kernels for Bayesian Optimization in Conditional Parameter Spaces

arXiv 1409.4011 ↗First posted 14 Sep 2014stat.MLcited by 16

Not on the map. Why?

Abstract (arXiv)

In practical Bayesian optimization, we must often search over structures with differing numbers of parameters. For instance, we may wish to search over neural network architectures with an unknown number of layers. To relate performance data gathered for different architectures, we define a new kernel for conditional parameter spaces that explicitly includes information about which parameters are relevant in a given structure. We show that this kernel improves model quality and Bayesian optimization results over several simpler baseline kernels.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Sep 2014.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 4; 2023: 3; 2024: 1; 2025: 2; 2026: 2.

16 papers.

Sort:

  1. 2021Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challengescited by 143
  2. 2023Neural Architecture Search: Insights from 1000 Paperscited by 136
  3. 2021How Powerful are Performance Predictors in Neural Architecture Search?cited by 88
  4. 2024Vanilla Bayesian Optimization Performs Great in High Dimensionscited by 87
  5. 2023PFNs4BO: In-Context Learning for Bayesian Optimizationcited by 70
  6. 2021Explaining Hyperparameter Optimization via Partial Dependence Plotscited by 35
  7. 2022Neural Architecture Search for Dense Prediction Tasks in Computer Visioncited by 11
  8. 2022Construction of Hierarchical Neural Architecture Search Spaces based on Context-free Grammarscited by 9
  9. 2021Scalable One-Pass Optimisation of High-Dimensional Weight-Update Hyperparameters by Implicit Differentiationcited by 7
  10. 2022Hybrid Parameter Search and Dynamic Model Selection for Mixed-Variable Bayesian Optimizationcited by 4
  11. 2023AutoML for Large Capacity Modeling of Meta's Ranking Systemscited by 4
  12. 2022GAAF: Searching Activation Functions for Binary Neural Networks through Genetic Algorithmcited by 1
4 more
  1. 2026Automated Kernel Discovery Towards Understanding High-dimensional Bayesian Optimizationcited by 1
  2. 2025Modeling All Response Surfaces in One for Conditional Search Spacescited by 0
  3. 2025Bayesian Optimization for Simultaneous Selection of Machine Learning Algorithms and Hyperparameters on Shared Latent Spacecited by 0
  4. 2026Conditional PED-ANOVA: Hyperparameter Importance in Hierarchical & Dynamic Search Spacescited by 0