mlsanityData to 7 Oct 2026

Bayesian Active Learning with Fully Bayesian Gaussian Processes

arXiv 2205.10186 ↗First posted 20 May 2022cs.LGcited by 26

Not on the map. Why?

Abstract (arXiv)

The bias-variance trade-off is a well-known problem in machine learning that only gets more pronounced the less available data there is. In active learning, where labeled data is scarce or difficult to obtain, neglecting this trade-off can cause inefficient and non-optimal querying, leading to unnecessary data labeling. In this paper, we focus on active learning with Gaussian Processes (GPs). For the GP, the bias-variance trade-off is made by optimization of the two hyperparameters: the length scale and noise-term. Considering that the optimal mode of the joint posterior of the hyperparameters is equivalent to the optimal bias-variance trade-off, we approximate this joint posterior and utilize it to design two new acquisition functions. The first one is a Bayesian variant of Query-by-Committee (B-QBC), and the second is an extension that explicitly minimizes the predictive variance through a Query by Mixture of Gaussian Processes (QB-MGP) formulation. Across six simulators, we empirically show that B-QBC, on average, achieves the best marginal likelihood, whereas QB-MGP achieves the best predictive performance. We show that incorporating the bias-variance trade-off in the acquisition functions mitigates unnecessary and expensive data labeling.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 13 of its 56 references that are in the map. First posted on arXiv 20 May 2022.

Builds on

13 of 56 references.

References · 13

  1. 2018GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Accelerationcited by 603
  2. 2019BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learningcited by 428
  3. 2018Pyro: Deep Universal Probabilistic Programmingcited by 379
  4. 2014Scalable Variational Gaussian Process Classificationcited by 226
  5. 2020Active Learning for Deep Gaussian Process Surrogatescited by 38
  6. 2016A Benchmark and Comparison of Active Learning for Logistic Regressioncited by 36
  7. 2018Pool-Based Sequential Active Learning for Regressioncited by 33
  8. 2019Approximate Inference for Fully Bayesian Gaussian Process Regressioncited by 31
  9. 2020Stacking for Non-mixing Bayesian Computations: The Curse and Blessing of Multimodal Posteriorscited by 29
  10. 2018Active Learning for Regression Using Greedy Samplingcited by 26
  11. 2019Active emulation of computer codes with Gaussian processes -- Application to remote sensingcited by 11
  12. 2021Entropy-based adaptive design for contour finding and estimating reliabilitycited by 10
1 more
  1. 2020Adaptive quadrature schemes for Bayesian inference via active learningcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 6; 2024: 6; 2025: 7; 2026: 6.

26 papers.

Sort:

  1. 2023Active learning for data streams: a surveycited by 29
  2. 2023Self-Correcting Bayesian Optimization through Bayesian Active Learningcited by 22
  3. 2023SOBER: Highly Parallel Bayesian Optimization and Bayesian Quadrature over Discrete and Mixed Spacescited by 7
  4. 2024Robust Gaussian Processes via Relevance Pursuitcited by 7
  5. 2023Adaptive Batch Sizes for Active Learning A Probabilistic Numerics Approachcited by 6
  6. 2024A Framework for Strategic Discovery of Credible Neural Network Surrogate Models under Uncertaintycited by 5
  7. 2023Hierarchical-Hyperplane Kernels for Actively Learning Gaussian Process Models of Nonstationary Systemscited by 4
  8. 2025Scalable Gaussian Processes with Latent Kronecker Structurecited by 4
  9. 2025X-Factor: Quality Is a Dataset-Intrinsic Propertycited by 3
  10. 2023Learning Power Flow with Confidence: A Probabilistic Guarantee Framework for Voltage Riskcited by 2
  11. 2024Bayesian Active Learning for Censored Regressioncited by 2
  12. 2024Practical Bayesian Algorithm Execution via Posterior Samplingcited by 2
14 more
  1. 2025Zero-Shot Performance Prediction for Probabilistic Scaling Lawscited by 2
  2. 2025Informed Initialization for Bayesian Optimization and Active Learningcited by 2
  3. 2022Targeted active learning for probabilistic modelscited by 1
  4. 2024Global Safe Sequential Learning via Efficient Knowledge Transfercited by 1
  5. 2024Future Aware Safe Active Learning of Time Varying Systems using Gaussian Processescited by 1
  6. 2026The Speed-up Factor: A Quantitative Multi-Iteration Active Learning Performance Metriccited by 1
  7. 2026Enhancing Bayesian Optimization and Active Learning Through Kernel Diversitycited by 1
  8. 2025Bayesian Active Learning By Distribution Disagreementcited by 0
  9. 2025BLADE: Bayesian Langevin Active Discovery with Replica Exchange for Identification of Complex Systemscited by 0
  10. 2025Designing an Optimal Sensor Network via Minimizing Information Losscited by 0
  11. 2026CAAL: Confidence-Aware Active Learning for Heteroscedastic Atmospheric Regressioncited by 0
  12. 2026Quality-Controlled Active Learning via Gaussian Processes for Robust Structure-Property Learning in Autonomous Microscopycited by 0
  13. 2026The Geometry of Saturation: Effective Rank Predicts When Labels Stop Helping in Few-Shot Classificationcited by 0
  14. 2026Online Adaptive Kernel Mixing for Gaussian Process Decision Makingcited by 0