mlsanityData to 7 Oct 2026

Emergence of Compositional Representations in Restricted Boltzmann Machines

arXiv 1611.06759 ↗First posted 21 Nov 2016physics.data-an cond-mat.dis-nn cs.LGcited by 23

Not on the map. Why?

Abstract (arXiv)

Extracting automatically the complex set of features composing real high-dimensional data is crucial for achieving high performance in machine--learning tasks. Restricted Boltzmann Machines (RBM) are empirically known to be efficient for this purpose, and to be able to generate distributed and graded representations of the data. We characterize the structural conditions (sparsity of the weights, low effective temperature, nonlinearities in the activation functions of hidden units, and adaptation of fields maintaining the activity in the visible layer) allowing RBM to operate in such a compositional phase. Evidence is provided by the replica analysis of an adequate statistical ensemble of random RBMs and by RBM trained on the handwritten digits dataset MNIST.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Nov 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 5; 2023: 8; 2024: 4; 2025: 2; 2026: 2.

23 papers.

Sort:

  1. 2021The emergence of a concept in shallow neural networkscited by 20
  2. 2022Learning a Restricted Boltzmann Machine using biased Monte Carlo samplingcited by 11
  3. 2022Disentangling representations in Restricted Boltzmann Machines without adversariescited by 11
  4. 2021Restricted Boltzmann Machines as Models of Interacting Variablescited by 9
  5. 2023Inferring effective couplings with Restricted Boltzmann Machinescited by 8
  6. 2023Fast and Functional Structured Data Generators Rooted in Out-of-Equilibrium Physicscited by 7
  7. 2022Quantifying Relevance in Learning and Inferencecited by 5
  8. 2023Unsupervised hierarchical clustering using the learning dynamics of RBMscited by 4
  9. 2024GenEFT: Understanding Statics and Dynamics of Model Generalization via Effective Theorycited by 4
  10. 2023Interpretable machine learning of amino acid patterns in proteins: a statistical ensemble approachcited by 3
  11. 2023Parallel Learning by Multitasking Neural Networkscited by 3
  12. 2024Fundamental operating regimes, hyper-parameter fine-tuning and glassiness: towards an interpretable replica-theory for trained restricted Boltzmann machinescited by 3
11 more
  1. 2023Compositionally Equivariant Representation Learningcited by 2
  2. 2024Modeling Structured Data Learning with Restricted Boltzmann Machines in the Teacher-Student Settingcited by 2
  3. 2022A developmental approach for training deep belief networkscited by 1
  4. 2023Statistical Mechanics of Learning via Reverberation in Bidirectional Associative Memoriescited by 1
  5. 2025On the role of non-linear latent features in bipartite generative neural networkscited by 1
  6. 2022Pattern reconstruction with restricted Boltzmann machinescited by 0
  7. 2023Unsupervised and Supervised learning by Dense Associative Memory under replica symmetry breakingcited by 0
  8. 2024Configuration Interaction Guided Sampling with Interpretable Restricted Boltzmann Machinecited by 0
  9. 2025Emergence of Nonequilibrium Latent Cycles in Unsupervised Generative Modelingcited by 0
  10. 2026Activation Functions, Statistics and Learning of Higher-Order Interactions in Restricted Boltzmann Machinescited by 0
  11. 2026Equilibrium Training of Energy-Based Models with Parallel Trajectory Temperingcited by 0