Multi-objective Hyperparameter Optimization in the Age of Deep Learning
Not on the map. Why?
Abstract (arXiv)
While Deep Learning (DL) experts often have prior knowledge about which hyperparameter settings yield strong performance, only few Hyperparameter Optimization (HPO) algorithms can leverage such prior knowledge and none incorporate priors over multiple objectives. As DL practitioners often need to optimize not just one but many objectives, this is a blind spot in the algorithmic landscape of HPO. To address this shortcoming, we introduce PriMO, the first HPO algorithm that can integrate multi-objective user beliefs. We show PriMO achieves state-of-the-art performance across 8 DL benchmarks in the multi-objective and single-objective setting, clearly positioning itself as the new go-to HPO algorithm for DL practitioners.
Key facts. Cited by 1 paper in the map (data through 13 Oct 2026). Builds on 32 of its 93 references that are in the map. First posted on arXiv 11 Nov 2025.
Builds on
32 of 93 references.
References · 32
- 2017Attention Is All You Needcited by 60,106
- 2015Deep Residual Learning for Image Recognitioncited by 51,950
- 2020Language Models are Few-Shot Learnerscited by 32,320
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,476
- 2016Wide Residual Networkscited by 3,063
- 2014OpenML: networked science in machine learningcited by 524
- 2020Efficient Content-Based Sparse Attention with Routing Transformerscited by 470
- 2018BOHB: Robust and Efficient Hyperparameter Optimization at Scalecited by 467
- 2013One Billion Word Benchmark for Measuring Progress in Statistical Language Modelingcited by 306
- 2015Non-stochastic Best Arm Identification and Hyperparameter Optimizationcited by 221
- 2018A System for Massively Parallel Hyperparameter Tuningcited by 201
- 2020Differentiable Expected Hypervolume Improvement for Parallel Multi-Objective Bayesian Optimizationcited by 177
20 more
- 2023Unexpected Improvements to Expected Improvement for Bayesian Optimizationcited by 145
- 2017Multi-fidelity Bayesian Optimisation with Continuous Approximationscited by 120
- 2018A Flexible Framework for Multi-Objective Bayesian Optimization using Random Scalarizationscited by 98
- 2015Predictive Entropy Search for Multi-objective Bayesian Optimizationcited by 88
- 2020Max-value Entropy Search for Multi-Objective Bayesian Optimization with Constraintscited by 81
- 2022πBO: Augmenting Acquisition Functions with User Beliefs for Bayesian Optimizationcited by 69
- 2021Pre-trained Gaussian Processes for Bayesian Optimizationcited by 63
- 2021YAHPO Gym -- An Efficient Multi-Objective Multi-Fidelity Benchmark for Hyperparameter Optimizationcited by 61
- 2020Random Hypervolume Scalarizations for Provable Multi-Objective Black Box Optimizationcited by 60
- 2023PriorBand: Practical Hyperparameter Optimization in the Age of Deep Learningcited by 31
- 2020Incorporating Expert Prior in Bayesian Optimisation via Space Warpingcited by 23
- 2021Multi-objective Asynchronous Successive Halvingcited by 23
- 2020Multi-Fidelity Multi-Objective Bayesian Optimization: An Output Space Entropy Search Approachcited by 22
- 2021Bag of Baselines for Multi-objective Joint Neural Architecture Search and Hyperparameter Optimizationcited by 19
- 2021A multi-objective perspective on jointly tuning hardware and hyperparameterscited by 12
- 2021Leveraging Trust for Joint Multi-Objective and Multi-Fidelity Optimizationcited by 7
- 2023Multi-Objective Optimization of Performance and Interpretability of Tabular Supervised Machine Learning Modelscited by 6
- 2020Bayesian Optimization with a Prior for the Optimumcited by 5
- 2020Information-Theoretic Multi-Objective Bayesian Optimization with Continuous Approximationscited by 3
- 2024LMEMs for post-hoc analysis of HPO Benchmarkingcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.