mlsanityData to 7 Oct 2026

Deep Latent State Space Models for Time-Series Generation

arXiv 2212.12749 ↗First posted 24 Dec 2022stat.ML cs.AI cs.LGcited by 37

On the map: State-space sequence models (S4, Mamba) → Time series forecasting with SSMs

Abstract (arXiv)

Methods based on ordinary differential equations (ODEs) are widely used to build generative models of time-series. In addition to high computational overhead due to explicitly computing hidden states recurrence, existing ODE-based models fall short in learning sequence data with sharp transitions - common in many real-world systems - due to numerical challenges during optimization. In this work, we propose LS4, a generative model for sequences with latent variables evolving according to a state space ODE to increase modeling capacity. Inspired by recent deep state space models (S4), we achieve speedups by leveraging a convolutional representation of LS4 which bypasses the explicit evaluation of hidden states. We show that LS4 significantly outperforms previous continuous-time generative models in terms of marginal distribution, classification, and prediction scores on real-world datasets in the Monash Forecasting Repository, and is capable of modeling highly stochastic data with sharp temporal transitions. LS4 sets state-of-the-art for continuous-time latent generative models, with significant improvement of mean squared error and tighter variational lower bounds on irregularly-sampled datasets, while also being x100 faster than other baselines on long sequences.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Placed in Time series forecasting with SSMs, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 38 of its 58 references that are in the map. First posted on arXiv 24 Dec 2022.

Builds on

38 of 58 references.

References · 38

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  5. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  6. 2013Auto-Encoding Variational Bayescited by 3,980
  7. 2018Neural Ordinary Differential Equationscited by 3,733
  8. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  9. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  10. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  11. 2020HiPPO: Recurrent Memory with Optimal Polynomial Projectionscited by 600
  12. 2021CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputationcited by 577
26 more
  1. 2020Neural Controlled Differential Equations for Irregular Time Seriescited by 501
  2. 2015Importance Weighted Autoencoderscited by 454
  3. 2022On the Parameterization and Initialization of Diagonal State Space Modelscited by 418
  4. 2016SeqGAN: Sequence Generative Adversarial Nets with Policy Gradientcited by 375
  5. 2015A Recurrent Latent Variable Model for Sequential Datacited by 362
  6. 2021Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecastingcited by 332
  7. 2020Scalable Gradients for Stochastic Differential Equationscited by 264
  8. 2021Monash Time Series Forecasting Archivecited by 256
  9. 2019GRU-ODE-Bayes: Continuous modeling of sporadically-observed time seriescited by 230
  10. 2022Generating Videos with Dynamics-aware Implicit Generative Adversarial Networkscited by 186
  11. 2019GP-VAE: Deep Probabilistic Time Series Imputationcited by 171
  12. 2022It's Raw! Audio Generation with State-Space Modelscited by 171
  13. 2021Neural SDEs as Infinite-Dimensional GANscited by 159
  14. 2020Dissecting Neural ODEscited by 150
  15. 2021CKConv: Continuous Kernel Convolution For Sequential Datacited by 117
  16. 2022Generating Long Videos of Dynamic Scenescited by 113
  17. 2020Stochastic Latent Residual Video Predictioncited by 106
  18. 2021Modeling Irregular Time Series with Continuous Recurrent Unitscited by 101
  19. 2021Stiff Neural Ordinary Differential Equationscited by 77
  20. 2020Adaptive Checkpoint Adjoint Method for Gradient Estimation in Neural ODEcited by 76
  21. 2014Learning Stochastic Recurrent Networkscited by 60
  22. 2014Variational Recurrent Auto-Encoderscited by 53
  23. 2020Modeling Continuous Stochastic Processes with Dynamic Normalizing Flowscited by 37
  24. 2021Parallelizing Legendre Memory Unit Trainingcited by 22
  25. 2021Differentiable Multiple Shooting Layerscited by 20
  26. 2019WATTNet: Learning to Trade FX via Hierarchical Spatio-Temporal Representation of Highly Multivariate Time Seriescited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 9; 2024: 11; 2025: 12; 2026: 5.

37 papers.

Sort:

By area:

  1. 2024Deep Time Series Models: A Comprehensive Survey and Benchmarkcited by 191
  2. 2023Functional Flow Matchingcited by 63
  3. 2023Simple Hardware-Efficient Long Convolutions for Sequence Modelingcited by 62
  4. 2023On the Constrained Time-Series Generation Problemcited by 52
  5. 2023Generative Modeling of Regular and Irregular Time Series Data via Koopman VAEscited by 48
  6. 2024Utilizing Image Transforms and Diffusion Models for Generative Modeling of Short and Long Time Seriescited by 42
  7. 2023Facing Off World Model Backbones: RNNs, Transformers, and S4cited by 31
  8. 2023TSGBench: Time Series Generation Benchmarkcited by 24
  9. 2025CodeBrain: Bridging Decoupled Tokenizer and Multi-Scale Architecture for EEG Foundation Modelcited by 23
  10. 2024Slot State Space Modelscited by 17
  11. 2023Conditional Generators for Limit Order Book Environments: Explainability, Challenges, and Robustnesscited by 14
  12. 2024Geometric Trajectory Diffusion Modelscited by 13
25 more
  1. 2023Revisiting Structured Variational Autoencoderscited by 11
  2. 2024KalMamba: Towards Efficient Probabilistic State Space Models for RL under Uncertaintycited by 9
  3. 2025Forging Time Series with Language: A Large Language Model Approach to Synthetic Data Generationcited by 6
  4. 2024Deep Bayesian Filter for Bayes-faithful Data Assimilationcited by 5
  5. 2024TimeBridge: Better Diffusion Prior Design with Bridge Models for Time Series Generationcited by 5
  6. 2024Sequential Order-Robust Mamba for Time Series Forecastingcited by 5
  7. 2024Representation Learning For Efficient Deep Multi-Agent Reinforcement Learningcited by 3
  8. 2024Parallelizing Autoregressive Generation with Variational State Space Modelscited by 3
  9. 2025CENTS: Generating synthetic electricity consumption time series for rare and unseen scenarioscited by 3
  10. 2025A Deep State Space Model for Rainfall-Runoff Simulationscited by 3
  11. 2025CTBench: Cryptocurrency Time Series Generation Benchmarkcited by 3
  12. 2023Neural Stochastic Differential Equations with Change Points: A Generative Adversarial Approachcited by 2
  13. 2025DS-Diffusion: Data Style-Guided Diffusion Model for Time-Series Generationcited by 2
  14. 2025Sequence Modeling with Spectral Mean Flowscited by 2
  15. 2025TSGDiff: Rethinking Synthetic Time Series Generation from a Pure Graph Perspectivecited by 2
  16. 2026MOSAIC: Modular Orchestration for Structured Agentic Intelligence and Compositioncited by 2
  17. 2024Variational Neural Stochastic Differential Equations with Change Pointscited by 1
  18. 2025WDformer: A Wavelet-based Differential Transformer Model for Time Series Forecastingcited by 1
  19. 2025Forecast-Then-Optimize Deep Learning Methodscited by 0
  20. 2025Denoising Refinement Diffusion Models for Simultaneous Generation of Multi-scale Mobile Network Trafficcited by 0
  21. 2025Latent-Autoregressive GP-VAE Language Modelcited by 0
  22. 2026Generation of synthetic delay time series for air transport applicationscited by 0
  23. 2026SL-S4Wave: Self-Supervised Learning of Physiological Waveforms with Structured State Space Modelscited by 0
  24. 2026UniWind: Toward Unified Day-Ahead Wind Power Forecasting via Physics-Informed State Routingcited by 0
  25. 2026Deep-MKV-TS: Path-Dependent McKean--Vlasov Control for Financial Time Series Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Effectively Modeling Time Series with Simple Discrete State Spaces
  2. 2022Interpretable Latent Variables in Deep State Space Models
  3. 2026Diffusion-Driven State Space Models
  4. 2025Time Series Generation Under Data Scarcity: A Unified Generative Modeling Approach
  5. 2024Deep Learning-based Approaches for State Space Models: A Selective Review
  6. 2024SDE: A Simplified and Disentangled Dependency Encoding Framework for State Space Models in Time Series Forecasting
  7. 2026Unconditional flow-based time series generation with equivariance-regularised latent spaces
  8. 2025StoxLSTM: A Stochastic Extended Long Short-Term Memory Network for Time Series Forecasting