mlsanityData to 7 Oct 2026

High-Dimensional Bayesian Optimization via Semi-Supervised Learning with Optimized Unlabeled Data Sampling

arXiv 2305.02614 ↗First posted 4 May 2023cs.LG cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

We introduce a novel semi-supervised learning approach, named Teacher-Student Bayesian Optimization (textttTSBO), integrating the teacher-student paradigm into BO to minimize expensive labeled data queries for the first time. textttTSBO incorporates a teacher model, an unlabeled data sampler, and a student model. The student is trained on unlabeled data locations generated by the sampler, with pseudo labels predicted by the teacher. The interplay between these three components implements a unique selective regularization to the teacher in the form of student feedback. This scheme enables the teacher to predict high-quality pseudo labels, enhancing the generalization of the GP surrogate model in the search space. To fully exploit textttTSBO, we propose two optimized unlabeled data samplers to construct effective student feedback that well aligns with the objective of Bayesian optimization. Furthermore, we quantify and leverage the uncertainty of the teacher-student model for the provision of reliable feedback to the teacher in the presence of risky pseudo-label predictions. textttTSBO demonstrates significantly improved sample-efficiency in several global optimization tasks under tight labeled data budgets.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 39 of its 71 references that are in the map. First posted on arXiv 4 May 2023.

Builds on

39 of 71 references.

References · 39

  1. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  2. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,080
  3. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,390
  4. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
  5. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,097
  6. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,057
  7. 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,054
  8. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 918
  9. 2016Automatic chemical design using a data-driven continuous representation of moleculescited by 762
  10. 2018A Tutorial on Bayesian Optimizationcited by 734
  11. 2018Junction Tree Variational Autoencoder for Molecular Graph Generationcited by 647
  12. 2021FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labelingcited by 537
27 more
  1. 2020Meta Pseudo Labelscited by 372
  2. 2019Scalable Global Optimization via Local Bayesian Optimizationcited by 359
  3. 2019ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoringcited by 349
  4. 2019NAS-Bench-101: Towards Reproducible Neural Architecture Searchcited by 340
  5. 2018GuacaMol: Benchmarking Models for De Novo Molecular Designcited by 294
  6. 2017Grammar Variational Autoencodercited by 283
  7. 2015Semi-Supervised Learning with Ladder Networkscited by 246
  8. 2018Optimization of Molecules via Deep Reinforcement Learningcited by 199
  9. 2013Bayesian Optimization in a Billion Dimensions via Random Embeddingscited by 178
  10. 2022Sample Efficiency Matters: A Benchmark for Practical Molecular Optimizationcited by 151
  11. 2021High-Dimensional Bayesian Optimization with Sparse Axis-Aligned Subspacescited by 139
  12. 2023SoftMatch: Addressing the Quantity-Quality Trade-off in Semi-supervised Learningcited by 133
  13. 2020Sample-Efficient Optimization in the Latent Space of Deep Generative Models via Weighted Retrainingcited by 121
  14. 2018Syntax-Directed Variational Autoencoder for Structured Datacited by 103
  15. 2020Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimizationcited by 89
  16. 2022Local Latent Space Bayesian Optimization over Structured Inputscited by 78
  17. 2021High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric Learningcited by 54
  18. 2021Improving black-box optimization in VAE latent space using decoder uncertaintycited by 51
  19. 2023Increasing the Scope as You Learn: Adaptive Bayesian Optimization in Nested Subspacescited by 50
  20. 2021Combining Latent Space and Structured Kernels for Bayesian Optimization over Combinatorial Spacescited by 45
  21. 2017Neural networks for topology optimizationcited by 39
  22. 2018Semi-supervised Deep Kernel Learning: Regression with Unlabeled Data by Minimizing Predictive Variancecited by 33
  23. 2019All SMILES Variational Autoencodercited by 15
  24. 2022Learning Where To Look -- Generative NAS is Surprisingly Efficientcited by 15
  25. 2020Semi-supervised Embedding Learning for High-dimensional Bayesian Optimizationcited by 11
  26. 2013Learning Non-Parametric Basis Independent Models from Point Queries via Low-Rank Methodscited by 6
  27. 2019Global Adversarial Attacks for Assessing Deep Learning Robustnesscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 1; 2025: 1; 2026: 1.

5 papers.

  1. 2024A survey and benchmark of high-dimensional Bayesian optimization of discrete sequencescited by 22
  2. 2023PG-LBO: Enhancing High-Dimensional Bayesian Optimization with Pseudo-Label and Gaussian Process Guidancecited by 9
  3. 2023Joint Composite Latent Space Bayesian Optimizationcited by 5
  4. 2025ChemBOMAS: Accelerated BO in Chemistry with LLM-Enhanced Multi-Agent Systemcited by 3
  5. 2026Optimization of time-consuming experimental conditions using pseudo-experimental data guided by adaptive polynomial regressioncited by 0