# SPATE-GAN: Improved Generative Modeling of Dynamic Spatio-Temporal Patterns with an Autoregressive Embedding Loss

arXiv: https://arxiv.org/abs/2109.15044 · HTML: https://mlsanity.com/paper/2109.15044/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 50 references that are in the map.
- First posted on arXiv 30 Sep 2021.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [MoCoGAN: Decomposing Motion and Content for Video Generation](https://mlsanity.com/paper/1707.04993/) (2017)
- [Learning Generative Models with Sinkhorn Divergences](https://mlsanity.com/paper/1706.00292/) (2017)
- [Revisiting Classifier Two-Sample Tests](https://mlsanity.com/paper/1610.06545/) (2016)
- [Towards Physics-informed Deep Learning for Turbulent Flow Prediction](https://mlsanity.com/paper/1911.08655/) (2019)
- [Presence-Only Geographical Priors for Fine-Grained Image Classification](https://mlsanity.com/paper/1906.05272/) (2019)
- [Multi-Scale Representation Learning for Spatial Feature Distributions using Grid Cells](https://mlsanity.com/paper/2003.00824/) (2020)
- [COT-GAN: Generating Sequential Data via Causal Optimal Transport](https://mlsanity.com/paper/2006.08571/) (2020)
- [Physics-informed deep learning for incompressible laminar flows](https://mlsanity.com/paper/2002.10558/) (2020)
- [DPM: A Novel Training Method for Physics-Informed Neural Networks in Extrapolation](https://mlsanity.com/paper/2012.02681/) (2020)
- [Geo-Aware Networks for Fine-Grained Recognition](https://mlsanity.com/paper/1906.01737/) (2019)
- [EL-GAN: Embedding Loss Driven Generative Adversarial Networks for Lane Detection](https://mlsanity.com/paper/1806.05525/) (2018)
- [JSI-GAN: GAN-Based Joint Super-Resolution and Inverse Tone-Mapping with Pixel-Wise Task-Specific Filters for UHD HDR Video](https://mlsanity.com/paper/1909.04391/) (2019)
- [Deep Compositional Spatial Models](https://mlsanity.com/paper/1906.02840/) (2019)
- [Emotion Understanding in Videos Through Body, Context, and Visual-Semantic Embedding Loss](https://mlsanity.com/paper/2010.16396/) (2020)
- [Auxiliary-task learning for geographic data with autoregressive embeddings](https://mlsanity.com/paper/2006.10461/) (2020)
- [CNN-based Patch Matching for Optical Flow with Thresholded Hinge Embedding Loss](https://mlsanity.com/paper/1607.08064/) (2016)
- [Generative modeling of spatio-temporal weather patterns with extreme event conditioning](https://mlsanity.com/paper/2104.12469/) (2021)
- [Augmenting correlation structures in spatial data using deep generative models](https://mlsanity.com/paper/1905.09796/) (2019)

## Built on by (most cited first, 13 of 13)

- [SatCLIP: Global, General-Purpose Location Embeddings with Satellite Imagery](https://mlsanity.com/paper/2311.17179/) (2023)
- [Mission Critical -- Satellite Data is a Distinct Modality in Machine Learning](https://mlsanity.com/paper/2402.01444/) (2024)
- [Foundation Models for Weather and Climate Data Understanding: A Comprehensive Survey](https://mlsanity.com/paper/2312.03014/) (2023)
- [Positional Encoder Graph Neural Networks for Geographic Data](https://mlsanity.com/paper/2111.10144/) (2021)
- [A Survey of Generative Techniques for Spatial-Temporal Data Mining](https://mlsanity.com/paper/2405.09592/) (2024)
- [GeoPointGAN: Synthetic Spatial Data with Local Label Differential Privacy](https://mlsanity.com/paper/2205.08886/) (2022)
- [AtmoDist: Self-supervised Representation Learning for Atmospheric Dynamics](https://mlsanity.com/paper/2202.01897/) (2022)
- [The Weisfeiler-Lehman Distance: Reinterpretation and Connection with GNNs](https://mlsanity.com/paper/2302.00713/) (2023)
- [When do World Models Successfully Learn Dynamical Systems?](https://mlsanity.com/paper/2507.04898/) (2025)
- [Reflections from the Workshop on AI-Assisted Decision Making for Conservation](https://mlsanity.com/paper/2307.08774/) (2023)
- [Towards Climate Variable Prediction with Conditioned Spatio-Temporal Normalizing Flows](https://mlsanity.com/paper/2311.06958/) (2023)
- [Capturing Unseen Spatial Heat Extremes Through Dependence-Aware Generative Modeling](https://mlsanity.com/paper/2507.09211/) (2025)
- [Generative Adversarial Learning from Deterministic Processes](https://mlsanity.com/paper/2605.18425/) (2026)

## Abstract (arXiv)

From ecology to atmospheric sciences, many academic disciplines deal with data characterized by intricate spatio-temporal complexities, the modeling of which often requires specialized approaches. Generative models of these data are of particular interest, as they enable a range of impactful downstream applications like simulation or creating synthetic training data. Recent work has highlighted the potential of generative adversarial nets (GANs) for generating spatio-temporal data. A new GAN algorithm COT-GAN, inspired by the theory of causal optimal transport (COT), was proposed in an attempt to better tackle this challenge. However, the task of learning more complex spatio-temporal patterns requires additional knowledge of their specific data structures. In this study, we propose a novel loss objective combined with COT-GAN based on an autoregressive embedding to reinforce the learning of spatio-temporal dynamics. We devise SPATE (spatio-temporal association), a new metric measuring spatio-temporal autocorrelation by using the deviance of observations from their expected values. We compute SPATE for real and synthetic data samples and use it to compute an embedding loss that considers space-time interactions, nudging the GAN to learn outputs that are faithful to the observed dynamics. We test this new objective on a diverse set of complex spatio-temporal patterns: turbulent flows, log-Gaussian Cox processes and global weather data. We show that our novel embedding loss improves performance without any changes to the architecture of the COT-GAN backbone, highlighting our model's increased capacity for capturing autoregressive structures. We also contextualize our work with respect to recent advances in physics-informed deep learning and interdisciplinary work connecting neural networks with geographic and geophysical sciences.