# Reducing the Computational Cost of Deep Generative Models with Binary Neural Networks

arXiv: https://arxiv.org/abs/2010.13476 · HTML: https://mlsanity.com/paper/2010.13476/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Oct 2020.

## Built on by (most cited first, 6 of 6)

- [On the Out-of-distribution Generalization of Probabilistic Image Modelling](https://mlsanity.com/paper/2109.02639/) (2021)
- [BiDM: Pushing the Limit of Quantization for Diffusion Models](https://mlsanity.com/paper/2412.05926/) (2024)
- [Partition and Code: learning how to compress graphs](https://mlsanity.com/paper/2107.01952/) (2021)
- [Fast Lossless Neural Compression with Integer-Only Discrete Flows](https://mlsanity.com/paper/2206.08869/) (2022)
- [Differentially-Private Data Synthetisation for Efficient Re-Identification Risk Control](https://mlsanity.com/paper/2212.00484/) (2022)
- [Automated Privacy-Preserving Techniques via Meta-Learning](https://mlsanity.com/paper/2406.16456/) (2024)

## Abstract (arXiv)

Deep generative models provide a powerful set of tools to understand real-world data. But as these models improve, they increase in size and complexity, so their computational cost in memory and execution time grows. Using binary weights in neural networks is one method which has shown promise in reducing this cost. However, whether binary neural networks can be used in generative models is an open problem. In this work we show, for the first time, that we can successfully train generative models which utilize binary neural networks. This reduces the computational cost of the models massively. We develop a new class of binary weight normalization, and provide insights for architecture designs of these binarized generative models. We demonstrate that two state-of-the-art deep generative models, the ResNet VAE and Flow++ models, can be binarized effectively using these techniques. We train binary models that achieve loss values close to those of the regular models but are 90%-94% smaller in size, and also allow significant speed-ups in execution time.