mlsanityData to 7 Oct 2026

Incorporating Unlabelled Data into Bayesian Neural Networks

arXiv 2304.01762 ↗First posted 4 Apr 2023cs.LG cs.AI stat.MLcited by 9

Not on the map. Why?

Abstract (arXiv)

Conventional Bayesian Neural Networks (BNNs) are unable to leverage unlabelled data to improve their predictions. To overcome this limitation, we introduce Self-Supervised Bayesian Neural Networks, which use unlabelled data to learn models with suitable prior predictive distributions. This is achieved by leveraging contrastive pretraining techniques and optimising a variational lower bound. We then show that the prior predictive distributions of self-supervised BNNs capture problem semantics better than conventional BNN priors. In turn, our approach offers improved predictive performance over conventional BNNs, especially in low-budget regimes.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 56 of its 98 references that are in the map. First posted on arXiv 4 Apr 2023.

Builds on

56 of 98 references.

References · 56

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  5. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  6. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  7. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  8. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  9. 2015Variational Inference with Normalizing Flowscited by 2,161
  10. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,393
  11. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  12. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
38 more
  1. 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 813
  2. 2017Deep Bayesian Active Learning with Image Datacited by 812
  3. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 672
  4. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 555
  5. 2019On Variational Bounds of Mutual Informationcited by 550
  6. 2014Semi-Supervised Learning with Deep Generative Modelscited by 494
  7. 2015Variational Dropout and the Local Reparameterization Trickcited by 489
  8. 2020Bayesian Deep Learning and a Probabilistic Perspective of Generalizationcited by 480
  9. 2015Deep Kernel Learningcited by 440
  10. 2020Understanding the Behaviour of Contrastive Losscited by 397
  11. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 359
  12. 2019On Mutual Information Maximization for Representation Learningcited by 315
  13. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 307
  14. 2021Contrastive Learning Inverts the Data Generating Processcited by 220
  15. 2018Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batchescited by 133
  16. 2019Functional Variational Bayesian Neural Networkscited by 127
  17. 2021Bayesian Neural Network Priors Revisitedcited by 121
  18. 2017Bayesian Compression for Deep Learningcited by 116
  19. 2021Scalable Marginal Likelihood Estimation for Model Selection in Deep Learningcited by 105
  20. 2021Repulsive Deep Ensembles are Bayesiancited by 103
  21. 2021Priors in Bayesian Deep Learning: A Reviewcited by 101
  22. 2016Stochastic Variational Deep Kernel Learningcited by 92
  23. 2020PACOH: Bayes-Optimal Meta-Learning with PAC-Guaranteescited by 85
  24. 2022Do Bayesian Neural Networks Need To Be Fully Stochastic?cited by 63
  25. 2020All You Need is a Good Functional Prior for Bayesian Deep Learningcited by 61
  26. 2018Learning Invariances using the Marginal Likelihoodcited by 51
  27. 2020Global inducing point variational posteriors for Bayesian neural networks and deep Gaussian processescited by 43
  28. 2022Pre-Train Your Loss: Easy Bayesian Transfer Learning with Informative Priorscited by 43
  29. 2022Invariance Learning in Deep Neural Networks with Differentiable Laplace Approximationscited by 42
  30. 2020Understanding Variational Inference in Function-Spacecited by 40
  31. 2021Data augmentation in Bayesian neural networks and the cold posterior effectcited by 36
  32. 2019Benchmarking the Neural Linear Model for Regressioncited by 31
  33. 2021Dangers of Bayesian Model Averaging under Covariate Shiftcited by 31
  34. 2020Capturing Label Characteristics in VAEscited by 30
  35. 2021Disentangling the Roles of Curation, Data-Augmentation and the Prior in the Cold Posterior Effectcited by 22
  36. 2020The Ridgelet Prior: A Covariance Function Approach to Prior Specification for Bayesian Neural Networkscited by 20
  37. 2021BNNpriors: A library for Bayesian neural network inference with different prior distributionscited by 20
  38. 2020Improving Transformation Invariance in Contrastive Representation Learningcited by 16

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 4; 2024: 5.

9 papers.

  1. 2024Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AIcited by 60
  2. 2023Beyond Deep Ensembles: A Large-Scale Evaluation of Bayesian Deep Learning under Distribution Shiftcited by 20
  3. 2024Gaussian Stochastic Weight Averaging for Bayesian Low-Rank Adaptation of Large Language Modelscited by 20
  4. 2023Improving Neural Additive Models with Bayesian Principlescited by 15
  5. 2023Promises and Pitfalls of the Linearized Laplace in Bayesian Optimizationcited by 10
  6. 2024Bayesian Neural Networks with Domain Knowledge Priorscited by 8
  7. 2024Optimization Proxies using Limited Labeled Data and Training Time -- A Semi-Supervised Bayesian Neural Network Approachcited by 2
  8. 2023Uncertainty in Graph Contrastive Learning with Bayesian Neural Networkscited by 0
  9. 2024Towards Dynamic Feature Acquisition on Medical Time Series by Maximizing Conditional Mutual Informationcited by 0