Three Learning Stages and Accuracy-Efficiency Tradeoff of Restricted Boltzmann Machines
Not on the map. Why?
Abstract (arXiv)
Restricted Boltzmann Machines (RBMs) offer a versatile architecture for unsupervised machine learning that can in principle approximate any target probability distribution with arbitrary accuracy. However, the RBM model is usually not directly accessible due to its computational complexity, and Markov-chain sampling is invoked to analyze the learned probability distribution. For training and eventual applications, it is thus desirable to have a sampler that is both accurate and efficient. We highlight that these two goals generally compete with each other and cannot be achieved simultaneously. More specifically, we identify and quantitatively characterize three regimes of RBM learning: independent learning, where the accuracy improves without losing efficiency; correlation learning, where higher accuracy entails lower efficiency; and degradation, where both accuracy and efficiency no longer improve or even deteriorate. These findings are based on numerical experiments and heuristic arguments.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 16 of its 72 references that are in the map. First posted on arXiv 2 Sep 2022.
Builds on
16 of 72 references.
References · 16
- 2018A mathematical theory of semantic development in deep neural networkscited by 182
- 2018A high-bias, low-variance introduction to Machine Learning for physicistscited by 108
- 2016Why does deep and cheap learning work so well?cited by 100
- 2015Application of Quantum Annealing to Training of Deep Neural Networkscited by 37
- 2017Mutual Information, Neural Networks and the Renormalization Groupcited by 32
- 2017Neural-Network Quantum States, String-Bond States, and Chiral Topological Statescited by 31
- 2021Equilibrium and non-Equilibrium regimes in the learning of Restricted Boltzmann Machinescited by 27
- 2018Restricted Boltzmann Machines: Introduction and Reviewcited by 14
- 2021Barriers and Dynamical Paths in Alternating Gibbs Sampling of Restricted Boltzmann Machinescited by 11
- 2018Accelerated physical emulation of Bayesian inference in spiking neural networkscited by 9
- 2016Stochastic inference with spiking neurons in the high-conductance statecited by 6
- 2015Hierarchical Models as Marginals of Hierarchical Modelscited by 4
4 more
- 2018Accelerating Deep Learning with Memcomputingcited by 4
- 2020Spiking neuromorphic chip learns entangled quantum statescited by 3
- 2020Mode-Assisted Unsupervised Learning of Restricted Boltzmann Machinescited by 2
- 2021Variational learning of quantum ground states on spiking neuromorphic hardwarecited by 2
Built on by
Citing papers per year: 2023: 2.
2 papers.