mlsanityData to 14 Oct 2026

Function-Space Regularization for Deep Bayesian Classification

arXiv 2307.06055 ↗First posted 12 Jul 2023cs.LG stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

Bayesian deep learning approaches assume model parameters to be latent random variables and infer posterior distributions to quantify uncertainty, increase safety and trust, and prevent overconfident and unpredictable behavior. However, weight-space priors are model-specific, can be difficult to interpret and are hard to specify. Instead, we apply a Dirichlet prior in predictive space and perform approximate function-space variational inference. To this end, we interpret conventional categorical predictions from stochastic neural network classifiers as samples from an implicit Dirichlet distribution. By adapting the inference, the same function-space prior can be combined with different models without affecting model architecture or size. We illustrate the flexibility and efficacy of such a prior with toy experiments and demonstrate scalability, improved uncertainty quantification and adversarial robustness with large-scale image classification experiments.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 48 of its 85 references that are in the map. First posted on arXiv 12 Jul 2023.

Builds on

48 of 85 references.

References · 48

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  5. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  6. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  7. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  8. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
  9. 2018Robustness May Be at Odds with Accuracycited by 880
  10. 2017Deep Bayesian Active Learning with Image Datacited by 819
  11. 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 819
  12. 2018Evidential Deep Learning to Quantify Classification Uncertaintycited by 732
36 more
  1. 2018Predictive Uncertainty Estimation via Prior Networkscited by 557
  2. 2015Variational Dropout and the Local Reparameterization Trickcited by 490
  3. 2020Bayesian Deep Learning and a Probabilistic Perspective of Generalizationcited by 480
  4. 2018Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Surveycited by 460
  5. 2020BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learningcited by 371
  6. 2015Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networkscited by 295
  7. 2014Scalable Variational Gaussian Process Classificationcited by 228
  8. 2018Randomized Prior Functions for Deep Reinforcement Learningcited by 219
  9. 2019Cyclical Stochastic Gradient MCMC for Bayesian Deep Learningcited by 203
  10. 2020Efficient and Scalable Bayesian Neural Nets with Rank-1 Factorscited by 171
  11. 2019Ensemble Distribution Distillationcited by 158
  12. 2019Functional Variational Bayesian Neural Networkscited by 127
  13. 2020Continual Deep Learning by Functional Regularisation of Memorable Pastcited by 105
  14. 2019Reverse KL-Divergence Training of Prior Networks: Improved Uncertainty and Adversarial Robustnesscited by 104
  15. 2021Priors in Bayesian Deep Learning: A Reviewcited by 101
  16. 2017Sticking the Landing: Simple, Lower-Variance Gradient Estimators for Variational Inferencecited by 98
  17. 2017Noisy Natural Gradient as Variational Inferencecited by 94
  18. 2018Uncertainty in Neural Networks: Approximately Bayesian Ensemblingcited by 94
  19. 2019Approximate Inference Turns Deep Networks into Gaussian Processescited by 89
  20. 2020Bayesian Deep Learning via Subnetwork Inferencecited by 87
  21. 2015On Sparse variational methods and the Kullback-Leibler divergence between stochastic processescited by 57
  22. 2020A statistical theory of cold posteriors in deep neural networkscited by 54
  23. 2018Dirichlet-based Gaussian Processes for Large-scale Calibrated Classificationcited by 51
  24. 2018A Spectral Approach to Gradient Estimation for Implicit Distributionscited by 45
  25. 2018Variational Implicit Processescited by 41
  26. 2019Function Space Particle Optimization for Bayesian Neural Networkscited by 41
  27. 2020Understanding Variational Inference in Function-Spacecited by 40
  28. 2018Natural Gradients in Practice: Non-Conjugate Variational Inference in Gaussian Process Modelscited by 38
  29. 2019Radial Bayesian Neural Networks: Beyond Discrete Support In Large-Scale Bayesian Deep Learningcited by 37
  30. 2020Being Bayesian about Categorical Probabilitycited by 35
  31. 2021The Gaussian Neural Processcited by 34
  32. 2020Fast Predictive Uncertainty for Classification with Bayesian Deep Networkscited by 31
  33. 2017Scalable Gaussian Processes with Billions of Inducing Inputs via Tensor Train Decompositioncited by 28
  34. 2018Efficient Gaussian Process Classification Using Polya-Gamma Data Augmentationcited by 17
  35. 2015Scalable Gaussian Process Classification via Expectation Propagationcited by 8
  36. 2019Scalable Training of Inference Networks for Gaussian-Process Modelscited by 8

Built on by

Not cited yet.