# Bayesian Pyramids: Identifiable Multilayer Discrete Latent Structure Models for Discrete Data

arXiv: https://arxiv.org/abs/2101.10373 · HTML: https://mlsanity.com/paper/2101.10373/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jan 2021.

## Built on by (most cited first, 16 of 16)

- [Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AI](https://mlsanity.com/paper/2402.00809/) (2024)
- [Identifiable Deep Generative Models via Sparse Decoding](https://mlsanity.com/paper/2110.10804/) (2021)
- [Histopathology Whole Slide Image Analysis with Heterogeneous Graph Representation Learning](https://mlsanity.com/paper/2307.04189/) (2023)
- [Learning Discrete Concepts in Latent Hierarchical Models](https://mlsanity.com/paper/2406.00519/) (2024)
- [I Predict Therefore I Am: Is Next Token Prediction Enough to Learn Human-Interpretable Concepts from Data?](https://mlsanity.com/paper/2503.08980/) (2025)
- [Bayesian Joint Additive Factor Models for Multiview Learning](https://mlsanity.com/paper/2406.00778/) (2024)
- [Learning Discrete Latent Variable Structures with Tensor Rank Conditions](https://mlsanity.com/paper/2406.07020/) (2024)
- [Differentiable Causal Discovery For Latent Hierarchical Causal Models](https://mlsanity.com/paper/2411.19556/) (2024)
- [Graphical Dirichlet Process for Clustering Non-Exchangeable Grouped Data](https://mlsanity.com/paper/2302.09111/) (2023)
- [Deep Discrete Encoders: Identifiable Deep Generative Models for Rich Data with Discrete Latent Layers](https://mlsanity.com/paper/2501.01414/) (2025)
- [Learning by Analogy: A Causal Framework for Composition Generalization](https://mlsanity.com/paper/2512.10669/) (2025)
- [Dependent Latent Class Models](https://mlsanity.com/paper/2205.08677/) (2022)
- [On Theoretical Identifiability of Discrete Latent Causal Graphical Models](https://mlsanity.com/paper/2505.18410/) (2025)
- [Learning discrete Bayesian networks with hierarchical Dirichlet shrinkage](https://mlsanity.com/paper/2509.13267/) (2025)
- [Beyond the Black Box: Identifiable Interpretation and Control in Generative Models via Causal Minimality](https://mlsanity.com/paper/2512.10720/) (2025)
- [Concept Component Analysis: A Principled Approach for Concept Extraction in LLMs](https://mlsanity.com/paper/2601.20420/) (2026)

## Abstract (arXiv)

High dimensional categorical data are routinely collected in biomedical and social sciences. It is of great importance to build interpretable parsimonious models that perform dimension reduction and uncover meaningful latent structures from such discrete data. Identifiability is a fundamental requirement for valid modeling and inference in such scenarios, yet is challenging to address when there are complex latent structures. In this article, we propose a class of identifiable multilayer (potentially deep) discrete latent structure models for discrete data, termed Bayesian pyramids. We establish the identifiability of Bayesian pyramids by developing novel transparent conditions on the pyramid-shaped deep latent directed graph. The proposed identifiability conditions can ensure Bayesian posterior consistency under suitable priors. As an illustration, we consider the two-latent-layer model and propose a Bayesian shrinkage estimation approach. Simulation results for this model corroborate the identifiability and estimability of model parameters. Applications of the methodology to DNA nucleotide sequence data uncover useful discrete latent features that are highly predictive of sequence types. The proposed framework provides a recipe for interpretable unsupervised learning of discrete data, and can be a useful alternative to popular machine learning methods.