# Modeling Documents with Deep Boltzmann Machines

arXiv: https://arxiv.org/abs/1309.6865 · HTML: https://mlsanity.com/paper/1309.6865/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Sep 2013.

## Built on by (most cited first, 10 of 10)

- [Restricted Boltzmann Machine and Deep Belief Network: Tutorial and Survey](https://mlsanity.com/paper/2107.12521/) (2021)
- [Coin Flipping Neural Networks](https://mlsanity.com/paper/2206.09182/) (2022)
- [MCDFN: Supply Chain Demand Forecasting via an Explainable Multi-Channel Data Fusion Network Model](https://mlsanity.com/paper/2405.15598/) (2024)
- [Fundamental operating regimes, hyper-parameter fine-tuning and glassiness: towards an interpretable replica-theory for trained restricted Boltzmann machines](https://mlsanity.com/paper/2406.09924/) (2024)
- [Modeling Structured Data Learning with Restricted Boltzmann Machines in the Teacher-Student Setting](https://mlsanity.com/paper/2410.16150/) (2024)
- [Deep Discrete Encoders: Identifiable Deep Generative Models for Rich Data with Discrete Latent Layers](https://mlsanity.com/paper/2501.01414/) (2025)
- [Character-level Chinese Backpack Language Models](https://mlsanity.com/paper/2310.12751/) (2023)
- [Saddle Hierarchy in Dense Associative Memory](https://mlsanity.com/paper/2508.19151/) (2025)
- [Back to Square One: Superhuman Performance in Chutes and Ladders Through Deep Neural Networks and Tree Search](https://mlsanity.com/paper/2104.00698/) (2021)
- [Explaining Machine Learning and Memorization with Statistical Mechanics](https://mlsanity.com/paper/2606.31110/) (2026)

## Abstract (arXiv)

We introduce a Deep Boltzmann Machine model suitable for modeling and extracting latent semantic representations from a large unstructured collection of documents. We overcome the apparent difficulty of training a DBM with judicious parameter tying. This parameter tying enables an efficient pretraining algorithm and a state initialization scheme that aids inference. The model can be trained just as efficiently as a standard Restricted Boltzmann Machine. Our experiments show that the model assigns better log probability to unseen data than the Replicated Softmax model. Features extracted from our model outperform LDA, Replicated Softmax, and DocNADE models on document retrieval and document classification tasks.