mlsanityData to 7 Oct 2026

α-PFN: Fast Entropy Search via In-Context Learning

arXiv 2606.07134 ↗First posted 5 Jun 2026cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Information-theoretic acquisition functions such as Entropy Search (ES) offer a principled exploration-exploitation framework for Bayesian optimization (BO). However, their practical implementation relies on complicated and slow approximations, i.e., a Monte Carlo estimation of the information gain. This complexity can introduce numerical errors and requires specialized, hand-crafted implementations. We propose a two-stage amortization strategy that learns to approximate entropy search-based acquisition functions using Prior-data Fitted Networks (PFNs) in a single forward pass. A first PFN is trained to be conditioned on information about the optima; second, the α-PFN is trained to predict the expected information gain by training on information gains measured with the first PFN. The α-PFN offers a flexible learned approximation, which replaces the complex heuristic approximations with a single forward pass per candidate, enabling rapid and extensible acquisition evaluation. Empirically, our approach is competitive with state-of-the-art entropy search implementations on synthetic and real-world benchmarks, while accelerating the different entropy search variants across all our experiments, with speed ups over 50x. Source code: https://github.com/automl/AlphaPFN.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 29 of its 54 references that are in the map. First posted on arXiv 5 Jun 2026.

Builds on

29 of 54 references.

References · 29

  1. 2018Conditional Neural Processescited by 421
  2. 2019Scalable Global Optimization via Local Bayesian Optimizationcited by 359
  3. 2021Transformers Can Do Bayesian Inferencecited by 338
  4. 2017Max-value Entropy Search for Efficient Bayesian Optimizationcited by 202
  5. 2014Predictive Entropy Search for Efficient Global Optimization of Black-box Functionscited by 201
  6. 2016Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasetscited by 131
  7. 2023ForecastPFN: Synthetically-Trained Zero-Shot Forecastingcited by 102
  8. 2021Multi-Objective Bayesian Optimization over High-Dimensional Search Spacescited by 87
  9. 2024Vanilla Bayesian Optimization Performs Great in High Dimensionscited by 87
  10. 2022Towards Learning Universal Hyperparameter Optimizers with Transformerscited by 81
  11. 2022Local Latent Space Bayesian Optimization over Structured Inputscited by 78
  12. 2021HPO-B: A Large-Scale Reproducible Benchmark for Black-Box HPO based on OpenMLcited by 75
17 more
  1. 2019Meta-Learning Acquisition Functions for Transfer Learning in Bayesian Optimizationcited by 70
  2. 2023PFNs4BO: In-Context Learning for Bayesian Optimizationcited by 70
  3. 2020Auto-PyTorch Tabular: Multi-Fidelity MetaLearning for Efficient and Robust AutoDLcited by 67
  4. 2022Joint Entropy Search for Maximally-Informed Bayesian Optimizationcited by 55
  5. 2021GIBBON: General-purpose Information-Based Bayesian OptimisatioNcited by 47
  6. 2022Joint Entropy Search for Multi-objective Bayesian Optimizationcited by 45
  7. 2023Efficient Bayesian Learning Curve Extrapolation using Prior-Data Fitted Networkscited by 37
  8. 2023End-to-End Meta-Bayesian Optimisation with Transformer Neural Processescited by 31
  9. 2024In-Context Freeze-Thaw Bayesian Optimization for Hyperparameter Optimizationcited by 26
  10. 2021Sequential- and Parallel- Constrained Max-value Entropy Search via Information Lower Boundcited by 23
  11. 2023Self-Correcting Bayesian Optimization through Bayesian Active Learningcited by 22
  12. 2024Amortized Probabilistic Conditioning for Optimization, Simulation and Inferencecited by 22
  13. 2022A General Recipe for Likelihood-free Bayesian Optimizationcited by 19
  14. 2025Distribution Transformers: Fast Approximate Bayesian Inference With On-The-Fly Prior Adaptationcited by 12
  15. 2025ALINE: Joint Amortization for Bayesian Inference and Active Data Acquisitioncited by 8
  16. 2022Rectified Max-Value Entropy Search for Bayesian Optimizationcited by 6
  17. 2025Amortized Safe Active Learning for Real-Time Data Acquisition: Pretrained Neural Policies From Simulated Nonparametric Functionscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026IQS-BO: In-Context Query Selection for Bayesian Optimisationcited by 0