mlsanityData to 7 Oct 2026

Bootstrap Your Own Variance

arXiv 2312.03213 ↗First posted 6 Dec 2023cs.LG stat.MLcited by 1

Not on the map. Why?

Abstract (arXiv)

Understanding model uncertainty is important for many applications. We propose Bootstrap Your Own Variance (BYOV), combining Bootstrap Your Own Latent (BYOL), a negative-free Self-Supervised Learning (SSL) algorithm, with Bayes by Backprop (BBB), a Bayesian method for estimating model posteriors. We find that the learned predictive std of BYOV vs. a supervised BBB model is well captured by a Gaussian distribution, providing preliminary evidence that the learned parameter posterior is useful for label free uncertainty estimation. BYOV improves upon the deterministic BYOL baseline (+2.83% test ECE, +1.03% test Brier) and presents better calibration and reliability when tested with various augmentations (eg: +2.4% test ECE, +1.2% test Brier for Salt & Pepper noise).

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 38 of its 55 references that are in the map. First posted on arXiv 6 Dec 2023.

Builds on

38 of 55 references.

References · 38

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2017On Calibration of Modern Neural Networkscited by 4,860
  6. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  7. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  8. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  9. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  10. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  11. 2016Training Deep Nets with Sublinear Memory Costcited by 825
  12. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 555
26 more
  1. 2015Weight Uncertainty in Neural Networkscited by 554
  2. 2019Using Self-Supervised Learning Can Improve Model Robustness and Uncertaintycited by 511
  3. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 359
  4. 2016Ladder Variational Autoencoderscited by 315
  5. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 307
  6. 2019Monte Carlo Gradient Estimation in Machine Learningcited by 281
  7. 2017VAE with a VampPriorcited by 247
  8. 2013Expectation Propagation for approximate Bayesian inferencecited by 172
  9. 2013A General Framework for Updating Belief Distributionscited by 172
  10. 2018Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Samplingcited by 166
  11. 2019Practical Deep Learning with Bayesian Principlescited by 158
  12. 2018Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batchescited by 133
  13. 2021Bayesian Neural Network Priors Revisitedcited by 121
  14. 2020BYOL works even without batch statisticscited by 95
  15. 2018Lightweight Probabilistic Deep Networkscited by 77
  16. 2022Do Bayesian Neural Networks Need To Be Fully Stochastic?cited by 63
  17. 2019Generalized Variational Inference: Three arguments for deriving new Posteriorscited by 57
  18. 2022Bayesian Model Selection, the Marginal Likelihood, and Generalizationcited by 56
  19. 2018Importance Weighting and Variational Inferencecited by 55
  20. 2020Regularization Matters: A Nonparametric Perspective on Overparametrized Neural Networkcited by 38
  21. 2016Natural-Parameter Networks: A Class of Probabilistic Neural Networkscited by 33
  22. 2023How to Scale Your EMAcited by 22
  23. 2017Variational Memory Addressing in Generative Modelscited by 17
  24. 2018The Kanerva Machine: A Generative Distributed Memorycited by 13
  25. 2021Kanerva++: extending The Kanerva Machine with differentiable, locally block allocated latent memorycited by 3
  26. 2022Benchmark for Uncertainty & Robustness in Self-Supervised Learningcited by 1

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Self-Supervised Learning with Gaussian Processescited by 0