# Learning Dynamic Generator Model by Alternating Back-Propagation Through Time

arXiv: https://arxiv.org/abs/1812.10587 · HTML: https://mlsanity.com/paper/1812.10587/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Dec 2018.

## Built on by (most cited first, 18 of 18)

- [Learning Generative Vision Transformer with Energy-Based Latent Space for Saliency Prediction](https://mlsanity.com/paper/2112.13528/) (2021)
- [Stochastic Image-to-Video Synthesis using cINNs](https://mlsanity.com/paper/2105.04551/) (2021)
- [Semi-Supervised Video Deraining with Dynamical Rain Generator](https://mlsanity.com/paper/2103.07939/) (2021)
- [Blind Image Super-resolution with Elaborate Degradation Modeling on Noise and Kernel](https://mlsanity.com/paper/2107.00986/) (2021)
- [Adaptive Multi-stage Density Ratio Estimation for Learning Latent Space Energy-based Model](https://mlsanity.com/paper/2209.08739/) (2022)
- [Deep Generative model with Hierarchical Latent Factors for Time Series Anomaly Detection](https://mlsanity.com/paper/2202.07586/) (2022)
- [Molecule Design by Latent Prompt Transformer](https://mlsanity.com/paper/2310.03253/) (2023)
- [A Tale of Two Latent Flows: Learning Latent Space Normalizing Flow with Short-run Langevin Flow for Approximate Inference](https://mlsanity.com/paper/2301.09300/) (2023)
- [Dynamic Texture Synthesis by Incorporating Long-range Spatial and Temporal Correlations](https://mlsanity.com/paper/2104.05940/) (2021)
- [Latent Space Energy-based Neural ODEs](https://mlsanity.com/paper/2409.03845/) (2024)
- [An Energy-Based Prior for Generative Saliency](https://mlsanity.com/paper/2204.08803/) (2022)
- [DTSGAN: Learning Dynamic Textures via Spatiotemporal Generative Adversarial Network](https://mlsanity.com/paper/2412.16948/) (2024)
- [Langevin Autoencoders for Learning Deep Latent Variable Models](https://mlsanity.com/paper/2209.07036/) (2022)
- [Likelihood-Based Generative Radiance Field with Latent Space Energy-Based Model for 3D-Aware Disentangled Image Representation](https://mlsanity.com/paper/2304.07918/) (2023)
- [Spatially-Variant Degradation Model for Dataset-free Super-resolution](https://mlsanity.com/paper/2407.08252/) (2024)
- [Deep Consensus Learning](https://mlsanity.com/paper/2103.08475/) (2021)
- [STANLEY: Stochastic Gradient Anisotropic Langevin Dynamics for Learning Energy-Based Models](https://mlsanity.com/paper/2310.12667/) (2023)
- [Molecule Design by Latent Prompt Transformer](https://mlsanity.com/paper/2402.17179/) (2024)

## Abstract (arXiv)

This paper studies the dynamic generator model for spatial-temporal processes such as dynamic textures and action sequences in video data. In this model, each time frame of the video sequence is generated by a generator model, which is a non-linear transformation of a latent state vector, where the non-linear transformation is parametrized by a top-down neural network. The sequence of latent state vectors follows a non-linear auto-regressive model, where the state vector of the next frame is a non-linear transformation of the state vector of the current frame as well as an independent noise vector that provides randomness in the transition. The non-linear transformation of this transition model can be parametrized by a feedforward neural network. We show that this model can be learned by an alternating back-propagation through time algorithm that iteratively samples the noise vectors and updates the parameters in the transition model and the generator model. We show that our training method can learn realistic models for dynamic textures and action patterns.