# None To Optima in Few Shots: Bayesian Optimization with MDP Priors

arXiv: https://arxiv.org/abs/2511.01006 · HTML: https://mlsanity.com/paper/2511.01006/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 55 references that are in the map.
- First posted on arXiv 2 Nov 2025.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Transformers Can Do Bayesian Inference](https://mlsanity.com/paper/2112.10510/) (2021)
- [On Kernelized Multi-armed Bandits](https://mlsanity.com/paper/1704.00445/) (2017)
- [Neural Architecture Search with Bayesian Optimisation and Optimal Transport](https://mlsanity.com/paper/1802.07191/) (2018)
- [Large Language Models to Enhance Bayesian Optimization](https://mlsanity.com/paper/2402.03921/) (2024)
- [Transformer Neural Processes: Uncertainty-Aware Meta Learning Via Sequence Modeling](https://mlsanity.com/paper/2207.04179/) (2022)
- [Practical Multi-fidelity Bayesian Optimization for Hyperparameter Tuning](https://mlsanity.com/paper/1903.04703/) (2019)
- [Towards Learning Universal Hyperparameter Optimizers with Transformers](https://mlsanity.com/paper/2205.13320/) (2022)
- [HPO-B: A Large-Scale Reproducible Benchmark for Black-Box HPO based on OpenML](https://mlsanity.com/paper/2106.06257/) (2021)
- [PFNs4BO: In-Context Learning for Bayesian Optimization](https://mlsanity.com/paper/2305.17535/) (2023)
- [Few-Shot Bayesian Optimization with Deep Kernel Surrogates](https://mlsanity.com/paper/2101.07667/) (2021)
- [Meta-Learning Acquisition Functions for Transfer Learning in Bayesian Optimization](https://mlsanity.com/paper/1904.02642/) (2019)
- [BayesNAS: A Bayesian Approach for Neural Architecture Search](https://mlsanity.com/paper/1905.04919/) (2019)
- [Pre-trained Gaussian Processes for Bayesian Optimization](https://mlsanity.com/paper/2109.08215/) (2021)
- [Learning search spaces for Bayesian optimization: Another view of hyperparameter transfer learning](https://mlsanity.com/paper/1909.12552/) (2019)
- [End-to-End Meta-Bayesian Optimisation with Transformer Neural Processes](https://mlsanity.com/paper/2305.15930/) (2023)
- [Practical Transfer Learning for Bayesian Optimization](https://mlsanity.com/paper/1802.02219/) (2018)
- [Transfer Learning with Gaussian Processes for Bayesian Optimization](https://mlsanity.com/paper/2111.11223/) (2021)
- [Reinforced Few-Shot Acquisition Function Learning for Bayesian Optimization](https://mlsanity.com/paper/2106.04335/) (2021)
- [Warm Starting Bayesian Optimization](https://mlsanity.com/paper/1608.03585/) (2016)
- [Hyperparameter Learning via Distributional Transfer](https://mlsanity.com/paper/1810.06305/) (2018)
- [Transfer Learning based Search Space Design for Hyperparameter Tuning](https://mlsanity.com/paper/2206.02511/) (2022)
- [End-to-End Learning of Deep Kernel Acquisition Functions for Bayesian Optimization](https://mlsanity.com/paper/2111.00639/) (2021)

## Built on by (most cited first, 3 of 3)

- [Efficient Adaptive Data Acquisition via Pretrained Belief Representations](https://mlsanity.com/paper/2606.25197/) (2026)
- [Multi-Objective Coverage via Constraint Active Search](https://mlsanity.com/paper/2602.15595/) (2026)
- [In-Context Learning for Latent Space Bayesian Optimization](https://mlsanity.com/paper/2606.09664/) (2026)

## Abstract (arXiv)

Bayesian Optimization (BO) is an efficient tool for optimizing black-box functions, but its theoretical guarantees typically hold in the asymptotic regime. In many critical real-world applications such as drug discovery or materials design, where each evaluation can be very costly and time-consuming, BO becomes impractical for many evaluations. In this paper, we introduce the Procedure-inFormed BO (ProfBO) algorithm, which solves black-box optimization with remarkably few function evaluations. At the heart of our algorithmic design are Markov Decision Process (MDP) priors that model optimization trajectories from related source tasks, thereby capturing procedural knowledge on efficient optimization. We embed these MDP priors into a prior-fitted neural network and employ model-agnostic meta-learning for fast adaptation to new target tasks. Experiments on real-world Covid and Cancer benchmarks and hyperparameter tuning tasks demonstrate that ProfBO consistently outperforms state-of-the-art methods by achieving high-quality solutions with significantly fewer evaluations, making it ready for practical deployment.