mlsanityData to 14 Oct 2026

ZeroShotOpt: Towards Zero-Shot Pretrained Models for Efficient Black-Box Optimization

arXiv 2510.03051 ↗First posted 3 Oct 2025cs.LG cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Global optimization of expensive, derivative-free black-box functions requires extreme sample efficiency. While Bayesian optimization (BO) is the current state-of-the-art, its performance hinges on surrogate and acquisition function hyper-parameters that are often hand-tuned and fail to generalize across problem landscapes. We present ZeroShotOpt, a general-purpose, pretrained model for continuous black-box optimization tasks ranging from 2D to 20D. Our approach leverages offline reinforcement learning on large-scale optimization trajectories collected from 12 BO variants. To scale pretraining, we generate millions of synthetic Gaussian process-based functions with diverse landscapes, enabling the model to learn transferable optimization policies. As a result, ZeroShotOpt achieves robust zero-shot generalization on a wide array of unseen benchmarks, matching or surpassing the sample efficiency of leading global optimizers, including BO, while also offering a reusable foundation for future extensions and improvements. Our open-source code, dataset, and model are available at: https://github.com/jamisonmeindl/zeroshotopt

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 34 of its 47 references that are in the map. First posted on arXiv 3 Oct 2025.

Builds on

34 of 47 references.

References · 34

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  4. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  5. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  6. 2016Learning to learn by gradient descent by gradient descentcited by 620
  7. 2014OpenML: networked science in machine learningcited by 526
  8. 2021SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimizationcited by 230
  9. 2021Learning to Optimize: A Primer and A Benchmarkcited by 208
  10. 2016Learning to Optimizecited by 195
  11. 2024An Introduction to Vision-Language Modelingcited by 178
  12. 2023Unexpected Improvements to Expected Improvement for Bayesian Optimizationcited by 149
22 more
  1. 2021Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020cited by 148
  2. 2024Large Language Models to Enhance Bayesian Optimizationcited by 135
  3. 2022Q-learning Decision Transformer: Leveraging Dynamic Programming for Conditional Sequence Modelling in Offline RLcited by 103
  4. 2020HEBO Pushing The Limits of Sample-Efficient Hyperparameter Optimisationcited by 83
  5. 2022Towards Learning Universal Hyperparameter Optimizers with Transformerscited by 81
  6. 2021HPO-B: A Large-Scale Reproducible Benchmark for Black-Box HPO based on OpenMLcited by 75
  7. 2023PFNs4BO: In-Context Learning for Bayesian Optimizationcited by 73
  8. 2019Meta-Learning Acquisition Functions for Transfer Learning in Bayesian Optimizationcited by 70
  9. 2023Elastic Decision Transformercited by 67
  10. 2021Pre-trained Gaussian Processes for Bayesian Optimizationcited by 63
  11. 2024Large Language Models As Evolution Strategiescited by 58
  12. 2022Generative Pretraining for Black-Box Optimizationcited by 36
  13. 2024OmniPred: Language Models as Universal Regressorscited by 34
  14. 2023End-to-End Meta-Bayesian Optimisation with Transformer Neural Processescited by 32
  15. 2021Reinforced Few-Shot Acquisition Function Learning for Bayesian Optimizationcited by 23
  16. 2018AutoPrognosis: Automated Clinical Prognostic Modeling via Bayesian Optimization with Structured Kernel Learningcited by 22
  17. 2024Reinforced In-Context Black-Box Optimizationcited by 14
  18. 2024The Vizier Gaussian Process Bandit Algorithmcited by 14
  19. 2024Pretrained Optimization Model for Zero-Shot Black Box Optimizationcited by 13
  20. 2019COCO: The Large Scale Black-Box Optimization Benchmarking (bbob-largescale) Test Suitecited by 12
  21. 2023OpenBox: A Python Toolkit for Generalized Black-box Optimizationcited by 9
  22. 2021AutoOED: Automated Optimal Experiment Design Platformcited by 3

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025SemanticOpt: Towards LLM-Based Semantic Black-Box Optimizationcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Optimizing the Unknown: Black Box Bayesian Optimization with Energy-Based Model and Reinforcement Learning
  2. 2026Improving Zero-Shot Offline RL via Behavioral Task Sampling
  3. 2025Posterior Inference with Diffusion Models for High-dimensional Black-box Optimization
  4. 2025A Unified Framework for Zero-Shot Reinforcement Learning
  5. 2024Robust Guided Diffusion for Offline Black-Box Optimization
  6. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
  7. 2025Provable Zero-Shot Generalization in Offline Reinforcement Learning
  8. 2025Towards Robust Zero-Shot Reinforcement Learning