mlsanityData to 7 Oct 2026

BOFormer: Learning to Solve Multi-Objective Bayesian Optimization via Non-Markovian RL

arXiv 2505.21974 ↗First posted 28 May 2025cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

Bayesian optimization (BO) offers an efficient pipeline for optimizing black-box functions with the help of a Gaussian process prior and an acquisition function (AF). Recently, in the context of single-objective BO, learning-based AFs witnessed promising empirical results given its favorable non-myopic nature. Despite this, the direct extension of these approaches to multi-objective Bayesian optimization (MOBO) suffer from the textithypervolume identifiability issue, which results from the non-Markovian nature of MOBO problems. To tackle this, inspired by the non-Markovian RL literature and the success of Transformers in language modeling, we present a generalized deep Q-learning framework and propose textitBOFormer, which substantiates this framework for MOBO via sequence modeling. Through extensive evaluation, we demonstrate that BOFormer constantly outperforms the benchmark rule-based and learning-based algorithms in various synthetic MOBO and real-world multi-objective hyperparameter optimization problems. We have made the source code publicly available to encourage further research in this direction.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 48 of its 76 references that are in the map. First posted on arXiv 28 May 2025.

Builds on

48 of 76 references.

References · 48

  1. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  3. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  4. 2015Prioritized Experience Replaycited by 1,070
  5. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
  6. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  7. 2016Automatic chemical design using a data-driven continuous representation of moleculescited by 762
  8. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  9. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 457
  10. 2018Conditional Neural Processescited by 421
  11. 2019Attentive Neural Processescited by 280
  12. 2021SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimizationcited by 229
36 more
  1. 2023MVImgNet: A Large-scale Dataset of Multi-view Imagescited by 219
  2. 2022Multi-Game Decision Transformerscited by 212
  3. 2017Max-value Entropy Search for Efficient Bayesian Optimizationcited by 202
  4. 2018Bayesian Model-Agnostic Meta-Learningcited by 194
  5. 2022Online Decision Transformercited by 194
  6. 2020Differentiable Expected Hypervolume Improvement for Parallel Multi-Objective Bayesian Optimizationcited by 176
  7. 2017Unifying PAC and Regret: Uniform PAC Bounds for Episodic Reinforcement Learningcited by 175
  8. 2016Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasetscited by 131
  9. 2023Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functionscited by 129
  10. 2019Reinforcement Learning Upside Down: Don't Predict Rewards -- Just Map Them to Actionscited by 124
  11. 2021Parallel Bayesian Optimization of Multiple Noisy Objectives with Expected Hypervolume Improvementcited by 115
  12. 2021Generalized Decision Transformer for Offline Hindsight Information Matchingcited by 105
  13. 2018A Flexible Framework for Multi-Objective Bayesian Optimization using Random Scalarizationscited by 98
  14. 2015Predictive Entropy Search for Multi-objective Bayesian Optimizationcited by 88
  15. 2021Multi-Objective Bayesian Optimization over High-Dimensional Search Spacescited by 87
  16. 2020Max-value Entropy Search for Multi-Objective Bayesian Optimization with Constraintscited by 81
  17. 2022Towards Learning Universal Hyperparameter Optimizers with Transformerscited by 81
  18. 2019Meta-Learning Acquisition Functions for Transfer Learning in Bayesian Optimizationcited by 70
  19. 2021YAHPO Gym -- An Efficient Multi-Objective Multi-Fidelity Benchmark for Hyperparameter Optimizationcited by 61
  20. 2021Reinforcement Learning, Bit by Bitcited by 59
  21. 2019Multi-objective Bayesian Optimization using Pareto-frontier Entropycited by 57
  22. 2022Joint Entropy Search for Maximally-Informed Bayesian Optimizationcited by 55
  23. 2020A Unifying View of Optimism in Episodic Reinforcement Learningcited by 50
  24. 2022Joint Entropy Search for Multi-objective Bayesian Optimizationcited by 45
  25. 2022Uncertainty-Aware Search Framework for Multi-Objective Bayesian Optimizationcited by 43
  26. 2021Simple Agent, Complex Environment: Efficient Reinforcement Learning with Agent Statescited by 42
  27. 2022Settling the Reward Hypothesiscited by 33
  28. 2023End-to-End Meta-Bayesian Optimisation with Transformer Neural Processescited by 31
  29. 2016Predictive Entropy Search for Multi-objective Bayesian Optimization with Constraintscited by 27
  30. 2021Reinforced Few-Shot Acquisition Function Learning for Bayesian Optimizationcited by 22
  31. 2013Multiobjective optimization using Gaussian process emulators via stepwise uncertainty reductioncited by 21
  32. 2013The Sample-Complexity of General Reinforcement Learningcited by 18
  33. 2021Output Space Entropy Search Framework for Multi-Objective Bayesian Optimizationcited by 13
  34. 2023RTDK-BO: High Dimensional Bayesian Optimization with Reinforced Transformer Deep kernelscited by 11
  35. 2016Nonparametric General Reinforcement Learningcited by 10
  36. 2019Bayesian Optimization for Multi-objective Optimization and Multi-point Searchcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 4.

9 papers.

  1. 2025ALINE: Joint Amortization for Bayesian Inference and Active Data Acquisitioncited by 8
  2. 2025In-Context Multi-Objective Optimizationcited by 5
  3. 2025FoMEMO: Towards Foundation Models for Expensive Multi-objective Optimizationcited by 2
  4. 2025GRPOformer: Advancing Hyperparameter Optimization via Group Relative Policy Optimizationcited by 2
  5. 2026Efficient Adaptive Data Acquisition via Pretrained Belief Representationscited by 2
  6. 2026In-Context Black-Box Optimization with Unreliable Feedbackcited by 1
  7. 2025Model-free Reinforcement Learning for Model-based Control: Towards Safe, Interpretable and Sample-efficient Agentscited by 0
  8. 2026Converge Then Diversify: Decoupling Convergence and Diversity in Multi-Objective Bayesian Optimisationcited by 0
  9. 2026IQS-BO: In-Context Query Selection for Bayesian Optimisationcited by 0