mlsanityData to 7 Oct 2026

Accelerating Bayesian Optimization for Biological Sequence Design with Denoising Autoencoders

arXiv 2203.12742 ↗First posted 23 Mar 2022cs.LG cs.NE q-bio.QMcited by 99

Not on the map. Why?

Abstract (arXiv)

Bayesian optimization (BayesOpt) is a gold standard for query-efficient continuous optimization. However, its adoption for drug design has been hindered by the discrete, high-dimensional nature of the decision variables. We develop a new approach (LaMBO) which jointly trains a denoising autoencoder with a discriminative multi-task Gaussian process head, allowing gradient-based optimization of multi-objective acquisition functions in the latent space of the autoencoder. These acquisition functions allow LaMBO to balance the explore-exploit tradeoff over multiple design rounds, and to balance objective tradeoffs by optimizing sequences at many different points on the Pareto frontier. We evaluate LaMBO on two small-molecule design tasks, and introduce new tasks optimizing in silico and in vitro properties of large-molecule fluorescent proteins. In our experiments LaMBO outperforms genetic optimizers and does not require a large pretraining corpus, demonstrating that BayesOpt is practical and effective for biological sequence design.

Key facts. Cited by 99 papers in the map (data through 7 Oct 2026). Builds on 43 of its 90 references that are in the map. First posted on arXiv 23 Mar 2022.

Builds on

43 of 90 references.

References · 43

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2016Automatic chemical design using a data-driven continuous representation of moleculescited by 762
  3. 2018A Tutorial on Bayesian Optimizationcited by 734
  4. 2018Junction Tree Variational Autoencoder for Molecular Graph Generationcited by 647
  5. 2018GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Accelerationcited by 603
  6. 2015Deep Kernel Learningcited by 440
  7. 2013Gaussian Processes for Big Datacited by 377
  8. 2019Scalable Global Optimization via Local Bayesian Optimizationcited by 359
  9. 2020ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Predictioncited by 332
  10. 2021Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Developmentcited by 258
  11. 2019Evaluating Protein Transfer Learning with TAPEcited by 229
  12. 2020pymoo: Multi-objective Optimization in Pythoncited by 196
31 more
  1. 2018Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinementcited by 194
  2. 2013Bayesian Optimization in a Billion Dimensions via Random Embeddingscited by 178
  3. 2020Differentiable Expected Hypervolume Improvement for Parallel Multi-Objective Bayesian Optimizationcited by 176
  4. 2021Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020cited by 146
  5. 2020Sample-Efficient Optimization in the Latent Space of Deep Generative Models via Weighted Retrainingcited by 121
  6. 2017Multi-fidelity Bayesian Optimisation with Continuous Approximationscited by 120
  7. 2021Parallel Bayesian Optimization of Multiple Noisy Objectives with Expected Hypervolume Improvementcited by 115
  8. 2017Constrained Bayesian Optimization with Noisy Experimentscited by 103
  9. 2016Stochastic Variational Deep Kernel Learningcited by 92
  10. 2019Practical Multi-fidelity Bayesian Optimization for Hyperparameter Tuningcited by 91
  11. 2021Multi-Objective Bayesian Optimization over High-Dimensional Search Spacescited by 87
  12. 2021Iterative Refinement Graph Neural Network for Antibody Sequence-Structure Co-designcited by 82
  13. 2017The Uncertainty Bellman Equation and Explorationcited by 81
  14. 2022Local Latent Space Bayesian Optimization over Structured Inputscited by 78
  15. 2019Augmenting Genetic Algorithms with Deep Neural Networks for Exploring the Chemical Spacecited by 66
  16. 2019Latent-Variable Non-Autoregressive Neural Machine Translation with Deterministic Inference Using a Delta Posteriorcited by 58
  17. 2021High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric Learningcited by 54
  18. 2021Deciphering antibody affinity maturation with language models and weakly supervised learningcited by 52
  19. 2020MIMOSA: Multi-constraint Molecule Sampling for Molecule Optimizationcited by 48
  20. 2021Combining Latent Space and Structured Kernels for Bayesian Optimization over Combinatorial Spacescited by 45
  21. 2021Amortized Tree Generation for Bottom-up Synthesis Planning and Synthesizable Molecular Designcited by 44
  22. 2020Efficient Nonmyopic Bayesian Optimization via One-Shot Multi-Step Treescited by 41
  23. 2020Bayesian Optimization of Risk Measurescited by 35
  24. 2021Bayesian Optimization with High-Dimensional Outputscited by 32
  25. 2020Population-Based Black-Box Optimization for Biological Sequence Designcited by 29
  26. 2020Autonomous discovery in the chemical sciences part II: Outlookcited by 26
  27. 2021Conditioning Sparse Variational Gaussian Processes for Online Decision-makingcited by 23
  28. 2022AntBO: Towards Real-World Automated Antibody Design with Combinatorial Bayesian Optimisationcited by 14
  29. 2015Unbounded Bayesian Optimization via Regularizationcited by 13
  30. 2021Optimizing High-Dimensional Physics Simulations via Composite Bayesian Optimizationcited by 6
  31. 2019Batched Stochastic Bayesian Optimization via Combinatorial Constraints Designcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 9; 2023: 16; 2024: 35; 2025: 27; 2026: 12.

99 papers.

Sort:

By area:

  1. 2023Protein Design with Guided Discrete Diffusioncited by 154
  2. 2022Multi-Objective GFlowNetscited by 100
  3. 2024Fine-Tuned Language Models Generate Stable Inorganic Materials as Textcited by 92
  4. 2024Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AIcited by 60
  5. 2023A Study of Bayesian Neural Network Surrogates for Bayesian Optimizationcited by 50
  6. 2024A Sober Look at LLMs for Material Discovery: Are They Actually Good for Bayesian Optimization Over Molecules?cited by 46
  7. 2022GAUCHE: A Library for Gaussian Processes in Chemistrycited by 40
  8. 2023AbDiffuser: Full-Atom Generation of in vitro Functioning Antibodiescited by 35
  9. 2024Full-Atom Peptide Design based on Multi-modal Flow Matchingcited by 34
  10. 2022Discovering Many Diverse Solutions with Bayesian Optimizationcited by 29
  11. 2022Bayesian Optimization with Conformal Prediction Setscited by 27
  12. 2024Conformal Validity Guarantees Exist for Any Data Distribution (and How to Find Them)cited by 26
38 more
  1. 2024A survey and benchmark of high-dimensional Bayesian optimization of discrete sequencescited by 22
  2. 2023Improving Protein Optimization with Smoothed Fitness Landscapescited by 20
  3. 2023The Behavior and Convergence of Local Bayesian Optimizationcited by 19
  4. 2025Latent Bayesian Optimization via Autoregressive Normalizing Flowscited by 19
  5. 2023Advancing Bayesian Optimization via Learning Correlated Latent Spacecited by 18
  6. 2023Multi-Fidelity Active Learning with GFlowNetscited by 15
  7. 2024PROUD: PaRetO-gUided Diffusion Model for Multi-objective Generationcited by 15
  8. 2024Concept Bottleneck Language Models For protein designcited by 15
  9. 2024Offline Multi-Objective Optimizationcited by 14
  10. 2025Offline Model-Based Optimization: Comprehensive Reviewcited by 14
  11. 2023Order-Preserving GFlowNetscited by 13
  12. 2024Robust Optimization in Protein Fitness Landscapes Using Reinforcement Learning in Latent Spacecited by 13
  13. 2024Improved Off-policy Reinforcement Learning in Biological Sequence Designcited by 13
  14. 2024Offline Model-Based Optimization by Learning to Rankcited by 13
  15. 2025Return of the Latent Space COWBOYS: Re-thinking the use of VAEs for Bayesian Optimisation of Structured Spacescited by 13
  16. 2024TAGMol: Target-Aware Gradient-guided Molecule Generationcited by 12
  17. 2025Preference-Guided Diffusion for Multi-Objective Offline Optimizationcited by 12
  18. 2022Generalizing Bayesian Optimization with Decision-theoretic Entropiescited by 11
  19. 2024Annealing Flow Generative Models Towards Sampling High-Dimensional and Multi-Modal Distributionscited by 11
  20. 2022Graph Neural Network Banditscited by 10
  21. 2024Bayesian Optimization of Antibodies Informed by a Generative Model of Evolving Sequencescited by 10
  22. 2025Large language models as uncertainty-calibrated optimizers for experimental discoverycited by 10
  23. 2023PG-LBO: Enhancing High-Dimensional Bayesian Optimization with Pseudo-Label and Gaussian Process Guidancecited by 9
  24. 2024Hotspot-Driven Peptide Design via Multi-Fragment Autoregressive Extensioncited by 9
  25. 2022PropertyDAG: Multi-objective Bayesian optimization of partially ordered, mixed-variable properties for biological sequence designcited by 8
  26. 2024Generative AI for Controllable Protein Sequence Design: A Surveycited by 8
  27. 2024Approximation-Aware Bayesian Optimizationcited by 8
  28. 2024Generalists vs. Specialists: Evaluating LLMs on Highly-Constrained Biophysical Sequence Optimization Taskscited by 8
  29. 2024Closed-Form Test Functions for Biophysical Sequence Optimization Algorithmscited by 7
  30. 2025Steering Generative Models with Experimental Data for Protein Fitness Optimizationcited by 7
  31. 2025Steering Protein Language Modelscited by 7
  32. 2023Biological Sequence Kernels with Guaranteed Flexibilitycited by 6
  33. 2024Variational Search Distributionscited by 6
  34. 2025Towards Universal Offline Black-Box Optimization via Learning Language Model Embeddingscited by 6
  35. 2023Joint Composite Latent Space Bayesian Optimizationcited by 5
  36. 2024Bayesian Optimization through Gaussian Cox Process Models for Spatio-temporal Datacited by 5
  37. 2025DualEquiNet: A Dual-Space Hierarchical Equivariant Network for Large Biomoleculescited by 5
  38. 2025AReUReDi: Annealed Rectified Updates for Refining Discrete Flows with Multi-Objective Guidancecited by 5

50 most cited of 99 All 99 (JSON)