mlsanityData to 7 Oct 2026

Centroid Approximation for Bootstrap: Improving Particle Quality at Inference

arXiv 2110.08720 ↗First posted 17 Oct 2021cs.LG stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

Bootstrap is a principled and powerful frequentist statistical tool for uncertainty quantification. Unfortunately, standard bootstrap methods are computationally intensive due to the need of drawing a large i.i.d. bootstrap sample to approximate the ideal bootstrap distribution; this largely hinders their application in large-scale machine learning, especially deep learning problems. In this work, we propose an efficient method to explicitly optimize a small set of high quality ``centroid'' points to better approximate the ideal bootstrap distribution. We achieve this by minimizing a simple objective function that is asymptotically equivalent to the Wasserstein distance to the ideal bootstrap distribution. This allows us to provide an accurate estimation of uncertainty with a small number of bootstrap centroids, outperforming the naive i.i.d. sampling approach. Empirically, we show that our method can boost the performance of bootstrap in a variety of applications.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 32 of its 67 references that are in the map. First posted on arXiv 17 Oct 2021.

Builds on

32 of 67 references.

References · 32

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  2. 2017On Calibration of Modern Neural Networkscited by 4,860
  3. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  4. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  5. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  6. 2016OpenAI Gymcited by 1,684
  7. 2016Variational Inference: A Review for Statisticianscited by 1,466
  8. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  9. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
  10. 2017Deep Neural Networks as Gaussian Processescited by 617
  11. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 555
  12. 2016Deep Exploration via Bootstrapped DQNcited by 518
20 more
  1. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 498
  2. 2017Snapshot Ensembles: Train 1, get M for freecited by 349
  3. 2014Stochastic Gradient Hamiltonian Monte Carlocited by 341
  4. 2018Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Samplingcited by 166
  5. 2018Out-of-Distribution Detection Using an Ensemble of Self Supervised Leave-out Classifierscited by 126
  6. 2018High-Quality Prediction Intervals for Deep Learning: A Distribution-Free, Ensembled Approachcited by 117
  7. 2020Masksembles for Uncertainty Estimationcited by 88
  8. 2019Adaptive, Distribution-Free Prediction Intervals for Deep Networkscited by 56
  9. 2020Predictive Inference Is Free with the Jackknife+-after-Bootstrapcited by 55
  10. 2018Stein Pointscited by 43
  11. 2019Bootstrapping Upper Confidence Boundcited by 36
  12. 2019Sparse Variational Inference: Bayesian Coresets from Scratchcited by 28
  13. 2020Prediction Intervals: Split Normal Mixture from Quality-Driven Deep Ensemblescited by 14
  14. 2017Variational Gaussian Dropout is not Bayesiancited by 13
  15. 2016An Efficient Minibatch Acceptance Test for Metropolis-Hastingscited by 11
  16. 2018Wasserstein Measure Coresetscited by 10
  17. 2020Bayesian Bootstrap Spike-and-Slab LASSOcited by 8
  18. 2019Mini-batch Metropolis-Hastings MCMC with Reversible SGLD Proposalcited by 6
  19. 2020Simultaneous Inference for Massive Data: Distributed Bootstrapcited by 4
  20. 2020Asymptotics of the Empirical Bootstrap Method Beyond Asymptotic Normalitycited by 3

Built on by

Not cited yet.