mlsanityData to 14 Oct 2026

Natural Evolutionary Search meets Probabilistic Numerics

arXiv 2507.07288 ↗First posted 9 Jul 2025cs.LG cs.NEcited by 2

Not on the map. Why?

Abstract (arXiv)

Zeroth-order local optimisation algorithms are essential for solving real-valued black-box optimisation problems. Among these, Natural Evolution Strategies (NES) represent a prominent class, particularly well-suited for scenarios where prior distributions are available. By optimising the objective function in the space of search distributions, NES algorithms naturally integrate prior knowledge during initialisation, making them effective in settings such as semi-supervised learning and user-prior belief frameworks. However, due to their reliance on random sampling and Monte Carlo estimates, NES algorithms can suffer from limited sample efficiency. In this paper, we introduce a novel class of algorithms, termed Probabilistic Natural Evolutionary Strategy Algorithms (ProbNES), which enhance the NES framework with Bayesian quadrature. We show that ProbNES algorithms consistently outperforms their non-probabilistic counterparts as well as global sample efficient methods such as Bayesian Optimisation (BO) or πBO across a wide range of tasks, including benchmark test functions, data-driven optimisation tasks, user-informed hyperparameter tuning tasks and locomotion tasks.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 18 of its 43 references that are in the map. First posted on arXiv 9 Jul 2025.

Builds on

18 of 43 references.

References · 18

  1. 2016Automatic chemical design using a data-driven continuous representation of moleculescited by 765
  2. 2016The CMA Evolution Strategy: A Tutorialcited by 377
  3. 2019Scalable Global Optimization via Local Bayesian Optimizationcited by 374
  4. 2020Sample-Efficient Optimization in the Latent Space of Deep Generative Models via Weighted Retrainingcited by 123
  5. 2021The Bayesian Learning Rulecited by 99
  6. 2022Local Latent Space Bayesian Optimization over Structured Inputscited by 79
  7. 2022πBO: Augmenting Acquisition Functions with User Beliefs for Bayesian Optimizationcited by 70
  8. 2014Input Warping for Bayesian Optimization of Non-stationary Functionscited by 67
  9. 2021High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric Learningcited by 54
  10. 2018Semi-supervised Deep Kernel Learning: Regression with Unlabeled Data by Minimizing Predictive Variancecited by 34
  11. 2015Probabilistic Integration: A Role in Statistical Computation?cited by 26
  12. 2023A General Framework for User-Guided Bayesian Optimizationcited by 22
6 more
  1. 2019Knowing The What But Not The Where in Bayesian Optimizationcited by 20
  2. 2022Fast Bayesian Inference with Batch Bayesian Quadrature via Kernel Recombinationcited by 20
  3. 2020Warm Starting CMA-ES for Hyperparameter Optimizationcited by 19
  4. 2019Meta-Surrogate Benchmarking for Hyperparameter Optimizationcited by 17
  5. 2023Adaptive Batch Sizes for Active Learning A Probabilistic Numerics Approachcited by 6
  6. 2024High-dimensional Bayesian Optimization via Covariance Matrix Adaptation Strategycited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Open-Ended Task Discovery via Bayesian Optimizationcited by 2
  2. 2026Regret Analysis of Guided Diffusion for Black-Box Optimization over Structured Inputscited by 1