# Generative Modeling of Regular and Irregular Time Series Data via Koopman VAEs

arXiv: https://arxiv.org/abs/2310.02619 · HTML: https://mlsanity.com/paper/2310.02619/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 77 references that are in the map.
- First posted on arXiv 4 Oct 2023.

## Builds on (most cited first, 30 of 49)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Are Transformers Effective for Time Series Forecasting?](https://mlsanity.com/paper/2205.13504/) (2022)
- [Generating Sequences With Recurrent Neural Networks](https://mlsanity.com/paper/1308.0850/) (2013)
- [Recurrent Neural Networks for Multivariate Time Series with Missing Values](https://mlsanity.com/paper/1606.01865/) (2016)
- [Neural Controlled Differential Equations for Irregular Time Series](https://mlsanity.com/paper/2005.08926/) (2020)
- [Deep learning for universal linear embeddings of nonlinear dynamics](https://mlsanity.com/paper/1712.09707/) (2017)
- [Tutorial on Variational Autoencoders](https://mlsanity.com/paper/1606.05908/) (2016)
- [Resurrecting Recurrent Neural Networks for Long Sequences](https://mlsanity.com/paper/2303.06349/) (2023)
- [A Recurrent Latent Variable Model for Sequential Data](https://mlsanity.com/paper/1506.02216/) (2015)
- [Real-valued (Medical) Time Series Generation with Recurrent Conditional GANs](https://mlsanity.com/paper/1706.02633/) (2017)
- [NIPS 2016 Tutorial: Generative Adversarial Networks](https://mlsanity.com/paper/1701.00160/) (2016)
- [Modern Koopman Theory for Dynamical Systems](https://mlsanity.com/paper/2102.12086/) (2021)
- [Scalable Gradients for Stochastic Differential Equations](https://mlsanity.com/paper/2001.01328/) (2020)
- [Are GANs Created Equal? A Large-Scale Study](https://mlsanity.com/paper/1711.10337/) (2017)
- [Adversarial Audio Synthesis](https://mlsanity.com/paper/1802.04208/) (2018)
- [Professor Forcing: A New Algorithm for Training Recurrent Networks](https://mlsanity.com/paper/1610.09038/) (2016)
- [Learning Koopman Invariant Subspaces for Dynamic Mode Decomposition](https://mlsanity.com/paper/1710.04340/) (2017)
- [TimeVAE: A Variational Auto-Encoder for Multivariate Time Series Generation](https://mlsanity.com/paper/2111.08095/) (2021)
- [C-RNN-GAN: Continuous recurrent neural networks with adversarial training](https://mlsanity.com/paper/1611.09904/) (2016)
- [Dynamical Variational Autoencoders: A Comprehensive Review](https://mlsanity.com/paper/2008.12595/) (2020)
- [Forecasting Sequential Data using Consistent Koopman Autoencoders](https://mlsanity.com/paper/2003.02236/) (2020)
- [Data-driven approximation of the Koopman generator: Model reduction, system identification, and control](https://mlsanity.com/paper/1909.10638/) (2019)
- [COT-GAN: Generating Sequential Data via Causal Optimal Transport](https://mlsanity.com/paper/2006.08571/) (2020)

## Built on by (most cited first, 30 of 49)

- [UniTS: A Unified Multi-Task Time Series Model](https://mlsanity.com/paper/2403.00131/) (2024)
- [Rethinking Irregular Time Series Forecasting: A Simple yet Effective Baseline](https://mlsanity.com/paper/2505.11250/) (2025)
- [Utilizing Image Transforms and Diffusion Models for Generative Modeling of Short and Long Time Series](https://mlsanity.com/paper/2410.19538/) (2024)
- [Population Aware Diffusion for Time Series Generation](https://mlsanity.com/paper/2501.00910/) (2025)
- [Time Series Generation Under Data Scarcity: A Unified Generative Modeling Approach](https://mlsanity.com/paper/2505.20446/) (2025)
- [LifeBench: A Benchmark for Long-Horizon Multi-Source Memory](https://mlsanity.com/paper/2603.03781/) (2026)
- [One-Step Offline Distillation of Diffusion-based Models via Koopman Modeling](https://mlsanity.com/paper/2505.13358/) (2025)
- [Towards Identifiability of Hierarchical Temporal Causal Representation Learning](https://mlsanity.com/paper/2510.18310/) (2025)
- [Sequential Disentanglement by Extracting Static Information From A Single Sequence Element](https://mlsanity.com/paper/2406.18131/) (2024)
- [Learning Physics for Unveiling Hidden Earthquake Ground Motions via Conditional Generative Modeling](https://mlsanity.com/paper/2407.15089/) (2024)
- [A Diffusion Model for Regular Time Series Generation from Irregular Data with Completion and Masking](https://mlsanity.com/paper/2510.06699/) (2025)
- [Reviving Life on the Edge: Joint Score-Based Graph Generation of Rich Edge Attributes](https://mlsanity.com/paper/2402.04046/) (2024)
- [First-Order Manifold Data Augmentation for Regression Learning](https://mlsanity.com/paper/2406.10914/) (2024)
- [TimeBridge: Better Diffusion Prior Design with Bridge Models for Time Series Generation](https://mlsanity.com/paper/2408.06672/) (2024)
- [TimeAutoDiff: A Unified Framework for Generation, Imputation, Forecasting, and Time-Varying Metadata Conditioning of Heterogeneous Time Series Tabular Data](https://mlsanity.com/paper/2406.16028/) (2024)
- [Analyzing Deep Transformer Models for Time Series Forecasting via Manifold Learning](https://mlsanity.com/paper/2410.13792/) (2024)
- [Balanced Neural ODEs: nonlinear model order reduction and Koopman operator approximations](https://mlsanity.com/paper/2410.10174/) (2024)
- [A Deep State Space Model for Rainfall-Runoff Simulations](https://mlsanity.com/paper/2501.14980/) (2025)
- [A Multi-Task Learning Approach to Linear Multivariate Forecasting](https://mlsanity.com/paper/2502.03571/) (2025)
- [MSDformer: Multi-scale Discrete Transformer For Time Series Generation](https://mlsanity.com/paper/2505.14202/) (2025)
- [CTBench: Cryptocurrency Time Series Generation Benchmark](https://mlsanity.com/paper/2508.02758/) (2025)
- [WaveletDiff: Multilevel Wavelet Diffusion For Time Series Generation](https://mlsanity.com/paper/2510.11839/) (2025)
- [TSGM: Regular and Irregular Time-series Generation using Score-based Generative Models](https://mlsanity.com/paper/2511.21335/) (2025)
- [Is Flow Matching Just Trajectory Replay for Sequential Data?](https://mlsanity.com/paper/2602.08318/) (2026)
- [DS-Diffusion: Data Style-Guided Diffusion Model for Time-Series Generation](https://mlsanity.com/paper/2509.18584/) (2025)
- [Hierarchical Koopman Diffusion: Fast Generation with Interpretable Diffusion Trajectory](https://mlsanity.com/paper/2510.12220/) (2025)
- [Sequence Modeling with Spectral Mean Flows](https://mlsanity.com/paper/2510.15366/) (2025)
- [Diff-MN: Diffusion Parameterized MoE-NCDE for Continuous Time Series Generation with Irregular Observations](https://mlsanity.com/paper/2601.13534/) (2026)
- [E4GEN: Event-level Explainable Extreme-Enhanced Time-series Generation](https://mlsanity.com/paper/2606.01634/) (2026)
- [Uniform Transformation: Refining Latent Representation in Variational Autoencoders](https://mlsanity.com/paper/2407.02681/) (2024)

## Abstract (arXiv)

Generating realistic time series data is important for many engineering and scientific applications. Existing work tackles this problem using generative adversarial networks (GANs). However, GANs are unstable during training, and they can suffer from mode collapse. While variational autoencoders (VAEs) are known to be more robust to the these issues, they are (surprisingly) less considered for time series generation. In this work, we introduce Koopman VAE (KoVAE), a new generative framework that is based on a novel design for the model prior, and that can be optimized for either regular and irregular training data. Inspired by Koopman theory, we represent the latent conditional prior dynamics using a linear map. Our approach enhances generative modeling with two desired features: (i) incorporating domain knowledge can be achieved by leveraging spectral tools that prescribe constraints on the eigenvalues of the linear map; and (ii) studying the qualitative behavior and stability of the system can be performed using tools from dynamical systems theory. Our results show that KoVAE outperforms state-of-the-art GAN and VAE methods across several challenging synthetic and real-world time series generation benchmarks. Whether trained on regular or irregular data, KoVAE generates time series that improve both discriminative and predictive metrics. We also present visual evidence suggesting that KoVAE learns probability density functions that better approximate the empirical ground truth distribution.