mlsanityData to 14 Oct 2026

High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric Learning

arXiv 2106.03609 ↗First posted 7 Jun 2021cs.LGcited by 54

Not on the map. Why?

Abstract (arXiv)

We introduce a method combining variational autoencoders (VAEs) and deep metric learning to perform Bayesian optimisation (BO) over high-dimensional and structured input spaces. By adapting ideas from deep metric learning, we use label guidance from the blackbox function to structure the VAE latent space, facilitating the Gaussian process fit and yielding improved BO performance. Importantly for BO problem settings, our method operates in semi-supervised regimes where only few labelled data points are available. We run experiments on three real-world tasks, achieving state-of-the-art results on the penalised logP molecule generation benchmark using just 3% of the labelled data required by previous approaches. As a theoretical contribution, we present a proof of vanishing regret for VAE BO.

Key facts. Cited by 54 papers in the map (data through 14 Oct 2026). Builds on 57 of its 127 references that are in the map. First posted on arXiv 7 Jun 2021.

Builds on

57 of 127 references.

References · 57

  1. 2013Auto-Encoding Variational Bayescited by 4,053
  2. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
  3. 2020Supervised Contrastive Learningcited by 2,829
  4. 2016Automatic chemical design using a data-driven continuous representation of moleculescited by 765
  5. 2018A Tutorial on Bayesian Optimizationcited by 744
  6. 2019Self-supervised Visual Feature Learning with Deep Neural Networks: A Surveycited by 657
  7. 2018Junction Tree Variational Autoencoder for Molecular Graph Generationcited by 652
  8. 2018BOHB: Robust and Efficient Hyperparameter Optimization at Scalecited by 470
  9. 2015Deep Kernel Learningcited by 445
  10. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 436
  11. 2014Deep metric learning using Triplet networkcited by 422
  12. 2019Scalable Global Optimization via Local Bayesian Optimizationcited by 374
38 more
  1. 2018GuacaMol: Benchmarking Models for De Novo Molecular Designcited by 297
  2. 2017Grammar Variational Autoencodercited by 285
  3. 2018Molecular Sets (MOSES): A Benchmarking Platform for Molecular Generation Modelscited by 275
  4. 2019Learning Trajectory Dependencies for Human Motion Predictioncited by 246
  5. 2018Neural Architecture Search with Bayesian Optimisation and Optimal Transportcited by 195
  6. 2020Proxy Anchor Loss for Deep Metric Learningcited by 195
  7. 2020CLEAR: Contrastive Learning for Sentence Representationcited by 186
  8. 2013Bayesian Optimization in a Billion Dimensions via Random Embeddingscited by 182
  9. 2020CERT: Contrastive Self-supervised Learning for Language Understandingcited by 169
  10. 2021Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020cited by 148
  11. 2015High Dimensional Bayesian Optimisation and Bandits via Additive Modelscited by 139
  12. 2020Sample-Efficient Optimization in the Latent Space of Deep Generative Models via Weighted Retrainingcited by 123
  13. 2018Deep Metric Learning with Hierarchical Triplet Losscited by 112
  14. 2017Deep Metric Learning with Angular Losscited by 107
  15. 2020Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimizationcited by 92
  16. 2019ChemBO: Bayesian Optimization of Small Organic Molecules with Synthesizable Recommendationscited by 77
  17. 2019BayesNAS: A Bayesian Approach for Neural Architecture Searchcited by 68
  18. 2014Input Warping for Bayesian Optimization of Non-stationary Functionscited by 67
  19. 2019Tuning Hyperparameters without Grad Students: Scalable and Robust Bayesian Optimisation with Dragonflycited by 67
  20. 2018BOCK : Bayesian Optimization with Cylindrical Kernelscited by 65
  21. 2013A Survey on Metric Learning for Feature Vectors and Structured Datacited by 58
  22. 2020BOSS: Bayesian Optimization over String Spacescited by 54
  23. 2018High Dimensional Bayesian Optimization Using Dropoutcited by 40
  24. 2017Neural networks for topology optimizationcited by 39
  25. 2019Deep Metric Learning Beyond Binary Supervisioncited by 36
  26. 2013Active Learning of Linear Embeddings for Gaussian Processescited by 34
  27. 2018Semi-supervised Deep Kernel Learning: Regression with Unlabeled Data by Minimizing Predictive Variancecited by 34
  28. 2020Mercer Features for Efficient Combinatorial Bayesian Optimizationcited by 27
  29. 2017On the choice of the low-dimensional domain for global optimization via random embeddingscited by 25
  30. 2020Good practices for Bayesian Optimization of high dimensional structured spacescited by 24
  31. 2019The Group Loss for Deep Metric Learningcited by 20
  32. 2018TVAE: Triplet-Based Variational Autoencoder using Metric Learningcited by 18
  33. 2020Gaussian Process Molecule Property Prediction with FlowMOcited by 18
  34. 2019All SMILES Variational Autoencodercited by 15
  35. 2019Achieving Robustness to Aleatoric Uncertainty with Heteroscedastic Bayesian Optimisationcited by 11
  36. 2019Bayesian Optimization in Variational Latent Spaces with Dynamic Compressioncited by 10
  37. 2020Are we Forgetting about Compositional Optimisers in Bayesian Optimisation?cited by 9
  38. 2020Interpretable Neural Architecture Search via Bayesian Optimisation with Weisfeiler-Lehman Kernelscited by 7

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 3; 2022: 12; 2023: 13; 2024: 7; 2025: 13; 2026: 6.

54 papers.

Sort:

By area:

  1. 2022Accelerating Bayesian Optimization for Biological Sequence Design with Denoising Autoencoderscited by 102
  2. 2022Local Latent Space Bayesian Optimization over Structured Inputscited by 79
  3. 2022GAUCHE: A Library for Gaussian Processes in Chemistrycited by 41
  4. 2023Are Random Decompositions all we need in High Dimensional Bayesian Optimisation?cited by 36
  5. 2021HyAR: Addressing Discrete-Continuous Action Reinforcement Learning via Hybrid Action Representationcited by 35
  6. 2022Discovering Many Diverse Solutions with Bayesian Optimizationcited by 31
  7. 2024A survey and benchmark of high-dimensional Bayesian optimization of discrete sequencescited by 23
  8. 2023The Behavior and Convergence of Local Bayesian Optimizationcited by 20
  9. 2025Latent Bayesian Optimization via Autoregressive Normalizing Flowscited by 19
  10. 2023Advancing Bayesian Optimization via Learning Correlated Latent Spacecited by 18
  11. 2022AntBO: Towards Real-World Automated Antibody Design with Combinatorial Bayesian Optimisationcited by 16
  12. 2023Multi-Fidelity Active Learning with GFlowNetscited by 15
38 more
  1. 2022Fast Bayesian Optimization of Needle-in-a-Haystack Problems using Zooming Memory-Based Initialization (ZoMBI)cited by 14
  2. 2025Return of the Latent Space COWBOYS: Re-thinking the use of VAEs for Bayesian Optimisation of Structured Spacescited by 14
  3. 2025Large language models as uncertainty-calibrated optimizers for experimental discoverycited by 13
  4. 2021Computationally Efficient High-Dimensional Bayesian Optimization via Variable Selectioncited by 12
  5. 2024Inversion-based Latent Bayesian Optimizationcited by 11
  6. 2023An Equivariant Generative Framework for Molecular Graph-Structure Co-Designcited by 10
  7. 2023PG-LBO: Enhancing High-Dimensional Bayesian Optimization with Pseudo-Label and Gaussian Process Guidancecited by 9
  8. 2022Conditional β-VAE for De Novo Molecular Generationcited by 8
  9. 2022Optimizing Training Trajectories in Variational Autoencoders via Latent Bayesian Optimization Approachcited by 7
  10. 2023Latent Space Bayesian Optimization with Latent Data Augmentation for Enhanced Explorationcited by 5
  11. 2023High-Dimensional Bayesian Optimization via Semi-Supervised Learning with Optimized Unlabeled Data Samplingcited by 5
  12. 2023Joint Composite Latent Space Bayesian Optimizationcited by 5
  13. 2026BayeSQP: Bayesian Optimization through Sequential Quadratic Programmingcited by 5
  14. 2024Crystal-LSBO: Automated Design of De Novo Crystals with Latent Space Bayesian Optimizationcited by 4
  15. 2024Active learning for affinity prediction of antibodiescited by 4
  16. 2024Safe Bayesian Optimization for the Control of High-Dimensional Embodied Systemscited by 4
  17. 2025Regret Analysis of Posterior Sampling-Based Expected Improvement for Bayesian Optimizationcited by 4
  18. 2022Sample-Efficient Optimisation with Probabilistic Transformer Surrogatescited by 3
  19. 2023Vecchia Gaussian Process Ensembles on Internal Representations of Deep Neural Networkscited by 3
  20. 2025Integrating Predictive and Generative Capabilities by Latent Space Design via the DKL-VAE Modelcited by 3
  21. 2026Purely Agent-Driven Black-Box Optimization for Biological Designcited by 3
  22. 2022Bayesian Optimization with Informative Covariancecited by 2
  23. 2022Structured Q-learning For Antibody Designcited by 2
  24. 2023A dynamic Bayesian optimized active recommender system for curiosity-driven Human-in-the-loop automated experimentscited by 2
  25. 2025Natural Evolutionary Search meets Probabilistic Numericscited by 2
  26. 2021A machine learning approach for fighting the curse of dimensionality in global optimizationcited by 1
  27. 2023Inverse Protein Folding Using Deep Bayesian Optimizationcited by 1
  28. 2024Conditional Latent Space Molecular Scaffold Optimization for Accelerated Molecular Designcited by 1
  29. 2024Dimensionality Reduction Techniques for Global Bayesian Optimisationcited by 1
  30. 2025Large Scale Multi-Task Bayesian Optimization with Large Language Modelscited by 1
  31. 2026Time-Aware Latent Space Bayesian Optimizationcited by 1
  32. 2026Regret Analysis of Guided Diffusion for Black-Box Optimization over Structured Inputscited by 1
  33. 2026Search at the Cost of Sampling: Nearly-Instant Latent Space Bayesian Optimizationcited by 1
  34. 2022Mold into a Graph: Efficient Bayesian Optimization over Mixed-Spacescited by 0
  35. 2023Applications of Gaussian Processes at Extreme Lengthscales: From Molecules to Black Holescited by 0
  36. 2025Modeling All Response Surfaces in One for Conditional Search Spacescited by 0
  37. 2025Covering Multiple Objectives with a Small Set of Solutions Using Bayesian Optimizationcited by 0
  38. 2025Sample-Efficient Optimisation over the Outputs of Generative Modelscited by 0

50 most cited of 54 All 54 (JSON)