Coin Flipping Neural Networks
Not on the map. Why?
Abstract (arXiv)
We show that neural networks with access to randomness can outperform deterministic networks by using amplification. We call such networks Coin-Flipping Neural Networks, or CFNNs. We show that a CFNN can approximate the indicator of a d-dimensional ball to arbitrary accuracy with only 2 layers and O(1) neurons, where a 2-layer deterministic network was shown to require Omega(ed) neurons, an exponential improvement (arXiv:1610.09887). We prove a highly non-trivial result, that for almost any classification problem, there exists a trivially simple network that solves it given a sufficiently powerful generator for the network's weights. Combining these results we conjecture that for most classification problems, there is a CFNN which solves them with higher accuracy or fewer neurons than any deterministic network. Finally, we verify our proofs experimentally using novel CFNN architectures on CIFAR10 and CIFAR100, reaching an improvement of 9.25% from the baseline.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 26 of its 46 references that are in the map. First posted on arXiv 18 Jun 2022.
Builds on
26 of 46 references.
References · 26
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
- 2013Auto-Encoding Variational Bayescited by 3,980
- 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
- 2014Conditional Generative Adversarial Netscited by 2,152
- 2021Alias-Free Generative Adversarial Networkscited by 1,181
- 2019An Introduction to Variational Autoencoderscited by 997
- 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 858
- 2016Tutorial on Variational Autoencoderscited by 404
- 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 337
- 2020Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Userscited by 336
14 more
- 2017The Expressive Power of Neural Networks: A View from the Widthcited by 318
- 2018Deep Learning in Spiking Neural Networkscited by 258
- 2016Benefits of depth in neural networkscited by 248
- 2015The Power of Depth for Feedforward Neural Networkscited by 181
- 2020A Tutorial on Learning With Bayesian Networkscited by 56
- 2020Hypermodels for Explorationcited by 37
- 2020Deep Randomized Neural Networkscited by 33
- 2018Stochastic Deep Networkscited by 13
- 2019Deep Neural Network Ensemblescited by 10
- 2013Modeling Documents with Deep Boltzmann Machinescited by 9
- 2019Stochastic Feedforward Neural Networks: Universal Approximationcited by 7
- 2017Boltzmann machines and energy-based modelscited by 5
- 2018On the Universal Approximation Property and Equivalence of Stochastic Computing-based Neural Networks and Binary Neural Networkscited by 4
- 2020Realization of Stochastic Neural Networks and Its Potential Applicationscited by 1
Built on by
Citing papers per year: 2023: 2; 2024: 1.
3 papers.