mlsanityData to 14 Oct 2026

Direct Regret Optimization in Bayesian Optimization

arXiv 2507.06529 ↗First posted 9 Jul 2025cs.LGcited by 0

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

Bayesian optimization (BO) is a powerful paradigm for optimizing expensive black-box functions. Traditional BO methods typically rely on separate hand-crafted acquisition functions and surrogate models for the underlying function, and often operate in a myopic manner. In this paper, we propose a novel direct regret optimization approach that jointly learns the optimal model and non-myopic acquisition by distilling from a set of candidate models and acquisitions, and explicitly targets minimizing the multi-step regret. Our framework leverages an ensemble of Gaussian Processes (GPs) with varying hyperparameters to generate simulated BO trajectories, each guided by an acquisition function drawn from a pool of conventional choices and terminated by a Bayesian early stop criterion. These trajectories train an end-to-end Decision Transformer that selects the next query so as to improve the ultimate objective, following a dense training sparse learning paradigm: the transformer is trained on abundant simulated data, while a limited number of real evaluations refine the GPs online. On synthetic and real-world benchmarks, our method attains the best or near-best final simple regret against standard, lookahead, trust-region and amortized BO baselines, with the largest gains in high-dimensional settings. Ablations attribute the gains jointly to region-of-interest filtering and the learned policy, and matched-budget comparisons against explicit two-step lookahead acquisitions show that the advantage is not shared by lookahead alone.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 37 of its 69 references that are in the map. First posted on arXiv 9 Jul 2025.

Builds on

37 of 69 references.

References · 37

  1. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  2. 2018Conditional Neural Processescited by 426
  3. 2019Scalable Global Optimization via Local Bayesian Optimizationcited by 374
  4. 2021Transformers Can Do Bayesian Inferencecited by 360
  5. 2017On Kernelized Multi-armed Banditscited by 352
  6. 2018Neural Processescited by 311
  7. 2019Attentive Neural Processescited by 283
  8. 2023Unexpected Improvements to Expected Improvement for Bayesian Optimizationcited by 149
  9. 2020Sample-Efficient Optimization in the Latent Space of Deep Generative Models via Weighted Retrainingcited by 123
  10. 2017Multi-fidelity Bayesian Optimisation with Continuous Approximationscited by 121
  11. 2021Deep Adaptive Design: Amortizing Sequential Bayesian Experimental Designcited by 106
  12. 2020Scalable Constrained Bayesian Optimizationcited by 84
25 more
  1. 2020HEBO Pushing The Limits of Sample-Efficient Hyperparameter Optimisationcited by 83
  2. 2022Towards Learning Universal Hyperparameter Optimizers with Transformerscited by 81
  3. 2023PFNs4BO: In-Context Learning for Bayesian Optimizationcited by 73
  4. 2019Meta-Learning Acquisition Functions for Transfer Learning in Bayesian Optimizationcited by 70
  5. 2023Statistical Foundations of Prior-Data Fitted Networkscited by 65
  6. 2021YAHPO Gym -- An Efficient Multi-Objective Multi-Fidelity Benchmark for Hyperparameter Optimizationcited by 62
  7. 2021Implicit Deep Adaptive Design: Policy-Based Experimental Design without Likelihoodscited by 61
  8. 2021Gaussian Process Bandit Optimization with Few Batchescited by 57
  9. 2019No-Regret Bayesian Optimization with Unknown Hyperparameterscited by 51
  10. 2022Constrained Efficient Global Optimization of Expensive Black-box Functionscited by 49
  11. 2022Optimizing Sequential Experimental Design with Deep Reinforcement Learningcited by 46
  12. 2020Efficient Nonmyopic Bayesian Optimization via One-Shot Multi-Step Treescited by 41
  13. 2015GLASSES: Relieving The Myopia Of Bayesian Optimisationcited by 38
  14. 2023End-to-End Meta-Bayesian Optimisation with Transformer Neural Processescited by 32
  15. 2021Reinforced Few-Shot Acquisition Function Learning for Bayesian Optimizationcited by 23
  16. 2024Amortized Probabilistic Conditioning for Optimization, Simulation and Inferencecited by 23
  17. 2023Self-Correcting Bayesian Optimization through Bayesian Active Learningcited by 22
  18. 2024Amortized Bayesian Experimental Design for Decision-Makingcited by 21
  19. 2021Deep Learning for Bayesian Optimization of Scientific Problems with High-Dimensional Structurecited by 15
  20. 2016Funneled Bayesian Optimization for Design, Tuning and Control of Autonomous Systemscited by 9
  21. 2023Learning Regions of Interest for Bayesian Optimization with Adaptive Level-Set Estimationcited by 8
  22. 2025PABBO: Preferential Amortized Black-Box Optimizationcited by 8
  23. 2023Combining Multi-Objective Bayesian Optimization with Reinforcement Learning for TinyMLcited by 6
  24. 2024EARL-BO: Reinforcement Learning for Multi-Step Lookahead, High-Dimensional Bayesian Optimizationcited by 6
  25. 2022Robust Bayesian optimization with reinforcement learned acquisition functionscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Nonparametric Bayesian Optimization for General Rewards
  2. 2026Amortized Acquisition Optimization for Bayesian Optimization with Variational Mutual Information
  3. 2026Improving Bayesian Optimization via Training-Aware Conditional Diffusion Models
  4. 2026Agentic Bayesian Optimization through Surrogate-Augmented Autoresearch
  5. 2026Bayesian Optimization in Linear Time
  6. 2026IQS-BO: In-Context Query Selection for Bayesian Optimisation
  7. 2025BOFormer: Learning to Solve Multi-Objective Bayesian Optimization via Non-Markovian RL
  8. 2025LLINBO: Trustworthy LLM-in-the-Loop Bayesian Optimization