# Deep Temporal Sigmoid Belief Networks for Sequence Modeling

arXiv: https://arxiv.org/abs/1509.07087 · HTML: https://mlsanity.com/paper/1509.07087/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Sep 2015.

## Built on by (most cited first, 14 of 14)

- [SDE Matching: Scalable and Simulation-Free Training of Latent Stochastic Differential Equations](https://mlsanity.com/paper/2502.02472/) (2025)
- [Amortized Reparametrization: Efficient and Scalable Variational Inference for Latent SDEs](https://mlsanity.com/paper/2312.10550/) (2023)
- [Latent Neural ODEs with Sparse Bayesian Multiple Shooting](https://mlsanity.com/paper/2210.03466/) (2022)
- [Scalable Inference in SDEs by Direct Matching of the Fokker-Planck-Kolmogorov Equation](https://mlsanity.com/paper/2110.15739/) (2021)
- [Modulated Neural ODEs](https://mlsanity.com/paper/2302.13262/) (2023)
- [Low-Rank Constraints for Fast Inference in Structured Models](https://mlsanity.com/paper/2201.02715/) (2022)
- [Neural Stochastic Flows: Solver-Free Modelling and Inference for SDE Solutions](https://mlsanity.com/paper/2510.25769/) (2025)
- [Latent Space Energy-based Neural ODEs](https://mlsanity.com/paper/2409.03845/) (2024)
- [Improved Constrained Generation by Bridging Pretrained Generative Models](https://mlsanity.com/paper/2603.06742/) (2026)
- [Negative-Binomial Randomized Gamma Markov Processes for Heterogeneous Overdispersed Count Time Series](https://mlsanity.com/paper/2402.18995/) (2024)
- [Prospective Messaging: Learning in Networks with Communication Delays](https://mlsanity.com/paper/2407.05494/) (2024)
- [DNS: Data-driven Nonlinear Smoother for Complex Model-free Process](https://mlsanity.com/paper/2602.08560/) (2026)
- [Integration Matters: Rollout-Based Training for Constrained Diffusion Models](https://mlsanity.com/paper/2607.14398/) (2026)
- [Gaussian Flow Dynamics: Simulation-Free Neural SDE Learning Beyond One-Time Marginals](https://mlsanity.com/paper/2610.04390/) (2026)

## Abstract (arXiv)

Deep dynamic generative models are developed to learn sequential dependencies in time-series data. The multi-layered model is designed by constructing a hierarchy of temporal sigmoid belief networks (TSBNs), defined as a sequential stack of sigmoid belief networks (SBNs). Each SBN has a contextual hidden state, inherited from the previous SBNs in the sequence, and is used to regulate its hidden bias. Scalable learning and inference algorithms are derived by introducing a recognition model that yields fast sampling from the variational posterior. This recognition model is trained jointly with the generative model, by maximizing its variational lower bound on the log-likelihood. Experimental results on bouncing balls, polyphonic music, motion capture, and text streams show that the proposed approach achieves state-of-the-art predictive performance, and has the capacity to synthesize various sequences.