# GSNs : Generative Stochastic Networks

arXiv: https://arxiv.org/abs/1503.05571 · HTML: https://mlsanity.com/paper/1503.05571/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Mar 2015.

## Built on by (most cited first, 9 of 9)

- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [Tackling the Generative Learning Trilemma with Denoising Diffusion GANs](https://mlsanity.com/paper/2112.07804/) (2021)
- [Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models](https://mlsanity.com/paper/2103.04922/) (2021)
- [Score-Based Generative Modeling with Critically-Damped Langevin Diffusion](https://mlsanity.com/paper/2112.07068/) (2021)
- [Truncated Diffusion Probabilistic Models and Diffusion-based Adversarial Auto-Encoders](https://mlsanity.com/paper/2202.09671/) (2022)
- [Fast Sampling via Discrete Non-Markov Diffusion Models with Predetermined Transition Time](https://mlsanity.com/paper/2312.09193/) (2023)
- [Policy Gradient With Serial Markov Chain Reasoning](https://mlsanity.com/paper/2210.06766/) (2022)
- [Learning Stationary Markov Processes with Contrastive Adjustment](https://mlsanity.com/paper/2303.05497/) (2023)
- [Evolutionary Computation as Natural Generative AI](https://mlsanity.com/paper/2510.08590/) (2025)

## Abstract (arXiv)

We introduce a novel training principle for probabilistic models that is an alternative to maximum likelihood. The proposed Generative Stochastic Networks (GSN) framework is based on learning the transition operator of a Markov chain whose stationary distribution estimates the data distribution. Because the transition distribution is a conditional distribution generally involving a small move, it has fewer dominant modes, being unimodal in the limit of small moves. Thus, it is easier to learn, more like learning to perform supervised function approximation, with gradients that can be obtained by back-propagation. The theorems provided here generalize recent work on the probabilistic interpretation of denoising auto-encoders and provide an interesting justification for dependency networks and generalized pseudolikelihood (along with defining an appropriate joint distribution and sampling mechanism, even when the conditionals are not consistent). We study how GSNs can be used with missing inputs and can be used to sample subsets of variables given the rest. Successful experiments are conducted, validating these theoretical results, on two image datasets and with a particular architecture that mimics the Deep Boltzmann Machine Gibbs sampler but allows training to proceed with backprop, without the need for layerwise pretraining.