mlsanityData to 7 Oct 2026

Entropy-regularized Gradient Estimators for Approximate Bayesian Inference

arXiv 2503.11964 ↗First posted 15 Mar 2025cs.LG stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

Effective uncertainty quantification is important for training modern predictive models with limited data, enhancing both accuracy and robustness. While Bayesian methods are effective for this purpose, they can be challenging to scale. When employing approximate Bayesian inference, ensuring the quality of samples from the posterior distribution in a computationally efficient manner is essential. This paper addresses the estimation of the Bayesian posterior to generate diverse samples by approximating the gradient flow of the Kullback-Leibler (KL) divergence and the cross entropy of the target approximation under the metric induced by the Stein Operator. It presents empirical evaluations on classification tasks to assess the method's performance and discuss its effectiveness for Model-Based Reinforcement Learning that uses uncertainty-aware network dynamics models.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 28 of its 35 references that are in the map. First posted on arXiv 15 Mar 2025.

Builds on

28 of 35 references.

References · 28

  1. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  2. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,194
  3. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
  4. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
  5. 2018Predictive Uncertainty Estimation via Prior Networkscited by 555
  6. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 555
  7. 2016Deep Exploration via Bootstrapped DQNcited by 518
  8. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 498
  9. 2020Bayesian Deep Learning and a Probabilistic Perspective of Generalizationcited by 480
  10. 2017Stochastic Gradient Descent as Approximate Bayesian Inferencecited by 298
  11. 2021Randomized Ensembled Double Q-Learning: Learning Fast Without a Modelcited by 278
  12. 2019Improving Adversarial Robustness via Promoting Ensemble Diversitycited by 213
16 more
  1. 2020SUNRISE: A Simple Unified Framework for Ensemble Learning in Deep Reinforcement Learningcited by 150
  2. 2019Functional Variational Bayesian Neural Networkscited by 127
  3. 2020On Linear Identifiability of Learned Representationscited by 103
  4. 2021Repulsive Deep Ensembles are Bayesiancited by 103
  5. 2019Better Exploration with Optimistic Actor-Criticcited by 88
  6. 2022Deep Ensembles Work, But Are They Necessary?cited by 63
  7. 2017Bayesian Optimization with Gradientscited by 59
  8. 2021Cooperative Exploration for Multi-Agent Deep Reinforcement Learningcited by 57
  9. 2017Message Passing Stein Variational Gradient Descentcited by 47
  10. 2018A Unified Particle-Optimization Framework for Scalable Bayesian Samplingcited by 44
  11. 2021Annealed Stein Variational Gradient Descentcited by 24
  12. 2020Why do classifier accuracies show linear trends under distribution shift?cited by 21
  13. 2023When are ensembles really effective?cited by 21
  14. 2023Pathologies of Predictive Diversity in Deep Ensemblescited by 16
  15. 2024(Implicit) Ensembles of Ensembles: Epistemic Uncertainty Collapse in Large Modelscited by 15
  16. 2022Model-based Lifelong Reinforcement Learning with Bayesian Explorationcited by 12

Built on by

Not cited yet.