# Disentangling representations in Restricted Boltzmann Machines without adversaries

arXiv: https://arxiv.org/abs/2206.11600 · HTML: https://mlsanity.com/paper/2206.11600/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 73 references that are in the map.
- First posted on arXiv 23 Jun 2022.

## Builds on (most cited first, 22 of 22)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [Interpreting the Latent Space of GANs for Semantic Face Editing](https://mlsanity.com/paper/1907.10786/) (2019)
- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [Self-Normalizing Neural Networks](https://mlsanity.com/paper/1706.02515/) (2017)
- [Towards Principled Methods for Training Generative Adversarial Networks](https://mlsanity.com/paper/1701.04862/) (2017)
- [AttGAN: Facial Attribute Editing by Only Changing What You Want](https://mlsanity.com/paper/1711.10678/) (2017)
- [Fader Networks: Manipulating Images by Sliding Attributes](https://mlsanity.com/paper/1706.00409/) (2017)
- [The Gaussian equivalence of generative models for learning with shallow neural networks](https://mlsanity.com/paper/2006.14709/) (2020)
- [Measuring Disentanglement: A Review of Metrics](https://mlsanity.com/paper/2012.09276/) (2020)
- [Structured Disentangled Representations](https://mlsanity.com/paper/1804.02086/) (2018)
- [Equilibrium and non-Equilibrium regimes in the learning of Restricted Boltzmann Machines](https://mlsanity.com/paper/2105.13889/) (2021)
- [Emergence of Compositional Representations in Restricted Boltzmann Machines](https://mlsanity.com/paper/1611.06759/) (2016)
- [Accurate and Conservative Estimates of MRF Log-likelihood using Reverse Annealing](https://mlsanity.com/paper/1412.8566/) (2014)
- [Disentangling Factors of Variation by Mixing Them](https://mlsanity.com/paper/1711.07410/) (2017)
- ['Place-cell' emergence and learning of invariant data with restricted Boltzmann machines: breaking and dynamical restoration of continuous symmetries in the weight space](https://mlsanity.com/paper/1912.12942/) (2019)
- [Learning Anonymized Representations with Adversarial Neural Networks](https://mlsanity.com/paper/1802.09386/) (2018)
- [The Accuracy of Restricted Boltzmann Machine Models of Ising Systems](https://mlsanity.com/paper/2004.12867/) (2020)

## Built on by (most cited first, 11 of 11)

- [Fast and Functional Structured Data Generators Rooted in Out-of-Equilibrium Physics](https://mlsanity.com/paper/2307.06797/) (2023)
- [Conditioned quantum-assisted deep generative surrogate for particle-calorimeter interactions](https://mlsanity.com/paper/2410.22870/) (2024)
- [A mathematical theory for understanding when abstract representations emerge in neural networks](https://mlsanity.com/paper/2510.09816/) (2025)
- [Interpretable machine learning of amino acid patterns in proteins: a statistical ensemble approach](https://mlsanity.com/paper/2303.15228/) (2023)
- [Fundamental operating regimes, hyper-parameter fine-tuning and glassiness: towards an interpretable replica-theory for trained restricted Boltzmann machines](https://mlsanity.com/paper/2406.09924/) (2024)
- [A simple probabilistic neural network for machine understanding](https://mlsanity.com/paper/2210.13179/) (2022)
- [Investigating the generative dynamics of energy-based neural networks](https://mlsanity.com/paper/2305.06745/) (2023)
- [Evolving Restricted Boltzmann Machine-Kohonen Network for Online Clustering](https://mlsanity.com/paper/2402.09167/) (2024)
- [Exploring the Energy Landscape of RBMs: Reciprocal Space Insights into Bosons, Hierarchical Learning and Symmetry Breaking](https://mlsanity.com/paper/2503.21536/) (2025)
- [Photonic restricted Boltzmann machine for content generation tasks](https://mlsanity.com/paper/2508.20472/) (2025)
- [Diluting Restricted Boltzmann Machines](https://mlsanity.com/paper/2511.00648/) (2025)

## Abstract (arXiv)

A goal of unsupervised machine learning is to build representations of complex high-dimensional data, with simple relations to their properties. Such disentangled representations make easier to interpret the significant latent factors of variation in the data, as well as to generate new data with desirable features. Methods for disentangling representations often rely on an adversarial scheme, in which representations are tuned to avoid discriminators from being able to reconstruct information about the data properties (labels). Unfortunately adversarial training is generally difficult to implement in practice. Here we propose a simple, effective way of disentangling representations without any need to train adversarial discriminators, and apply our approach to Restricted Boltzmann Machines (RBM), one of the simplest representation-based generative models. Our approach relies on the introduction of adequate constraints on the weights during training, which allows us to concentrate information about labels on a small subset of latent variables. The effectiveness of the approach is illustrated with four examples: the CelebA dataset of facial images, the two-dimensional Ising model, the MNIST dataset of handwritten digits, and the taxonomy of protein families. In addition, we show how our framework allows for analytically computing the cost, in terms of log-likelihood of the data, associated to the disentanglement of their representations.