# Double-Bayesian Learning

arXiv: https://arxiv.org/abs/2410.12984 · HTML: https://mlsanity.com/paper/2410.12984/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 21 references that are in the map.
- First posted on arXiv 16 Oct 2024.

## Builds on (most cited first, 5 of 5)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Rethinking the Hyperparameters for Fine-tuning](https://mlsanity.com/paper/2002.11770/) (2020)
- [The Golden Ratio of Learning and Momentum](https://mlsanity.com/paper/2006.04751/) (2020)
- [A Dual Process Model for Optimizing Cross Entropy in Neural Networks](https://mlsanity.com/paper/2104.13277/) (2021)

## Built on by (most cited first, 1 of 1)

- [Training Neural Networks with Optimal Double-Bayesian Learning](https://mlsanity.com/paper/2605.20009/) (2026)

## Abstract (arXiv)

Contemporary machine learning methods will try to approach the Bayes error, as it is the lowest possible error any model can achieve. This paper postulates that any decision is composed of not one but two Bayesian decisions and that decision-making is, therefore, a double-Bayesian process. The paper shows how this duality implies intrinsic uncertainty in decisions and how it incorporates explainability. The proposed approach understands that Bayesian learning is tantamount to finding a base for a logarithmic function measuring uncertainty, with solutions being fixed points. Furthermore, following this approach, the golden ratio describes possible solutions satisfying Bayes' theorem. The double-Bayesian framework suggests using a learning rate and momentum weight with values similar to those used in the literature to train neural networks with stochastic gradient descent.