# Identifiability of Product of Experts Models

arXiv: https://arxiv.org/abs/2310.09397 · HTML: https://mlsanity.com/paper/2310.09397/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 27 references that are in the map.
- First posted on arXiv 13 Oct 2023.

## Builds on (most cited first, 10 of 10)

- [DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts](https://mlsanity.com/paper/2105.03023/) (2021)
- [Multimodal Conditional Image Synthesis with Product-of-Experts GANs](https://mlsanity.com/paper/2112.05130/) (2021)
- [Restricted Boltzmann Machines: Introduction and Review](https://mlsanity.com/paper/1806.07066/) (2018)
- [Source Identification for Mixtures of Product Distributions](https://mlsanity.com/paper/2012.14540/) (2020)
- [Causal Inference Despite Limited Global Confounding via Mixture Models](https://mlsanity.com/paper/2112.11602/) (2021)
- [The Sparse Hausdorff Moment Problem, with Application to Topic Models](https://mlsanity.com/paper/2007.08101/) (2020)
- [Dimension of Marginals of Kronecker Product Models](https://mlsanity.com/paper/1511.03570/) (2015)
- [Mixtures and products in two graphical models](https://mlsanity.com/paper/1709.05276/) (2017)
- [Efficient Algorithms for Sparse Moment Problems without Separation](https://mlsanity.com/paper/2207.13008/) (2022)
- [Discrete Restricted Boltzmann Machines](https://mlsanity.com/paper/1301.3529/) (2013)

## Built on by (most cited first, 3 of 3)

- [Compositional Generative Inverse Design](https://mlsanity.com/paper/2401.13171/) (2024)
- [Learning Discrete Latent Variable Structures with Tensor Rank Conditions](https://mlsanity.com/paper/2406.07020/) (2024)
- [Evolutionary Computation as Natural Generative AI](https://mlsanity.com/paper/2510.08590/) (2025)

## Abstract (arXiv)

Product of experts (PoE) are layered networks in which the value at each node is an AND (or product) of the values (possibly negated) at its inputs. These were introduced as a neural network architecture that can efficiently learn to generate high-dimensional data which satisfy many low-dimensional constraints -- thereby allowing each individual expert to perform a simple task. PoEs have found a variety of applications in learning. We study the problem of identifiability of a product of experts model having a layer of binary latent variables, and a layer of binary observables that are iid conditional on the latents. The previous best upper bound on the number of observables needed to identify the model was exponential in the number of parameters. We show: (a) When the latents are uniformly distributed, the model is identifiable with a number of observables equal to the number of parameters (and hence best possible). (b) In the more general case of arbitrarily distributed latents, the model is identifiable for a number of observables that is still linear in the number of parameters (and within a factor of two of best-possible). The proofs rely on root interlacing phenomena for some special three-term recurrences.