mlsanityData to 7 Oct 2026

Variational Search Distributions

arXiv 2409.06142 ↗First posted 10 Sep 2024stat.ML cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

We develop VSD, a method for conditioning a generative model of discrete, combinatorial designs on a rare desired class by efficiently evaluating a black-box (e.g. experiment, simulation) in a batch sequential manner. We call this task active generation; we formalize active generation's requirements and desiderata, and formulate a solution via variational inference. VSD uses off-the-shelf gradient based optimization routines, can learn powerful generative models for desirable designs, and can take advantage of scalable predictive models. We derive asymptotic convergence rates for learning the true conditional generative distribution of designs with certain configurations of our method. After illustrating the generative model on images, we empirically demonstrate that VSD can outperform existing baseline methods on a set of real sequence-design problems in various protein and DNA/RNA engineering tasks.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 49 of its 93 references that are in the map. First posted on arXiv 10 Sep 2024.

Builds on

49 of 93 references.

References · 49

  1. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  2. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  3. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
  4. 2016Automatic chemical design using a data-driven continuous representation of moleculescited by 762
  5. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 732
  6. 2017The Implicit Bias of Gradient Descent on Separable Datacited by 649
  7. 2018Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizoncited by 612
  8. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 498
  9. 2017On Kernelized Multi-armed Banditscited by 348
  10. 2013Black Box Variational Inferencecited by 317
  11. 2019Monte Carlo Gradient Estimation in Machine Learningcited by 281
  12. 2021On the Origin of Implicit Regularization in Stochastic Gradient Descentcited by 183
37 more
  1. 2022Biological Sequence Design with GFlowNetscited by 172
  2. 2020On Information Gain and Regret Bounds in Gaussian Process Banditscited by 156
  3. 2023Protein Design with Guided Discrete Diffusioncited by 154
  4. 2020Implicit Gradient Regularizationcited by 145
  5. 2020On the linearity of large non-linear models: when and why the tangent kernel is constantcited by 129
  6. 2022A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Modelscited by 128
  7. 2020Sample-Efficient Optimization in the Latent Space of Deep Generative Models via Weighted Retrainingcited by 121
  8. 2020Infinite attention: NNGP and NTK for deep attention networkscited by 117
  9. 2019Conditioning by adaptive sampling for robust designcited by 110
  10. 2022Accelerating Bayesian Optimization for Biological Sequence Design with Denoising Autoencoderscited by 99
  11. 2022Design-Bench: Benchmarks for Data-Driven Offline Model-Based Optimizationcited by 96
  12. 2021Conservative Objective Models for Effective Offline Model-Based Optimizationcited by 87
  13. 2020Deep Neural Tangent Kernel and Laplace Kernel Have the Same RKHScited by 74
  14. 2022Formal Algorithms for Transformerscited by 70
  15. 2020AdaLead: A simple and robust adaptive greedy search algorithm for sequence designcited by 63
  16. 2017The reparameterization trick for acquisition functionscited by 61
  17. 2016Truncated Variance Reduction: A Unified Approach to Bayesian Optimization and Level-Set Estimationcited by 40
  18. 2022Bayesian Optimization over Discrete and Mixed Spaces via Probabilistic Reparameterizationcited by 35
  19. 2023BayesDAG: Gradient-Based Posterior Inference for Causal Discoverycited by 33
  20. 2018Design by adaptive samplingcited by 31
  21. 2021BORE: Bayesian Optimization by Density-Ratio Estimationcited by 26
  22. 2022Characterizing the Spectrum of the NTK via a Power Series Expansioncited by 22
  23. 2024A survey and benchmark of high-dimensional Bayesian optimization of discrete sequencescited by 22
  24. 2023Kernelized Reinforcement Learning with Order Optimal Regret Boundscited by 20
  25. 2023Improving Protein Optimization with Smoothed Fitness Landscapescited by 20
  26. 2022A General Recipe for Likelihood-free Bayesian Optimizationcited by 19
  27. 2021Non-asymptotic approximations of neural networks by Gaussian processescited by 18
  28. 2017Streaming kernel regression with provably adaptive mean, variance, and regularizationcited by 15
  29. 2023Posterior Sampling-Based Bayesian Optimization with Tighter Bayesian Regret Boundscited by 15
  30. 2018Singular Value Decomposition of Operators on Reproducing Kernel Hilbert Spacescited by 14
  31. 2022Batch Bayesian optimisation via density-ratio estimation with guaranteescited by 9
  32. 2023Learning Regions of Interest for Bayesian Optimization with Adaptive Level-Set Estimationcited by 8
  33. 2024Closed-Form Test Functions for Biophysical Sequence Optimization Algorithmscited by 7
  34. 2015Discovering Valuable Items from Massive Datacited by 6
  35. 2018Stochastic Variational Optimizationcited by 5
  36. 2019A view of Estimation of Distribution Algorithms through the lens of Expectation-Maximizationcited by 5
  37. 2024A Continuous Relaxation for Discrete Bayesian Optimizationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 4; 2026: 1.

6 papers.

  1. 2024Generalists vs. Specialists: Evaluating LLMs on Highly-Constrained Biophysical Sequence Optimization Taskscited by 8
  2. 2025Variational Transdimensional Inferencecited by 2
  3. 2025Generative Bayesian Optimization: Generative Models as Acquisition Functionscited by 2
  4. 2025Amortized Active Generation of Pareto Setscited by 1
  5. 2026Kernel-based guarantees for nonlinear parametric models in Bayesian optimizationcited by 1
  6. 2025Sample-Efficient Optimisation over the Outputs of Generative Modelscited by 0