# Hierarchical Models as Marginals of Hierarchical Models

arXiv: https://arxiv.org/abs/1508.03606 · HTML: https://mlsanity.com/paper/1508.03606/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Aug 2015.

## Built on by (most cited first, 4 of 4)

- [Learning Gaussian-Bernoulli RBMs using Difference of Convex Functions Optimization](https://mlsanity.com/paper/2102.06228/) (2021)
- [Three Learning Stages and Accuracy-Efficiency Tradeoff of Restricted Boltzmann Machines](https://mlsanity.com/paper/2209.00873/) (2022)
- [Supermodular Rank: Set Function Decomposition and Optimization](https://mlsanity.com/paper/2305.14632/) (2023)
- [Configuration Interaction Guided Sampling with Interpretable Restricted Boltzmann Machine](https://mlsanity.com/paper/2409.06146/) (2024)

## Abstract (arXiv)

We investigate the representation of hierarchical models in terms of marginals of other hierarchical models with smaller interactions. We focus on binary variables and marginals of pairwise interaction models whose hidden variables are conditionally independent given the visible variables. In this case the problem is equivalent to the representation of linear subspaces of polynomials by feedforward neural networks with soft-plus computational units. We show that every hidden variable can freely model multiple interactions among the visible variables, which allows us to generalize and improve previous results. In particular, we show that a restricted Boltzmann machine with less than [ 2(log(v)+1) / (v+1) ] 2v-1 hidden binary variables can approximate every distribution of v visible binary variables arbitrarily well, compared to 2v-1-1 from the best previously known result.