# Time-series Transformer Generative Adversarial Networks

arXiv: https://arxiv.org/abs/2205.11164 · HTML: https://mlsanity.com/paper/2205.11164/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 55 references that are in the map.
- First posted on arXiv 23 May 2022.

## Builds on (most cited first, 30 of 33)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Modeling Tabular data using Conditional GAN](https://mlsanity.com/paper/1907.00503/) (2019)
- [Generating Sequences With Recurrent Neural Networks](https://mlsanity.com/paper/1308.0850/) (2013)
- [f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization](https://mlsanity.com/paper/1606.00709/) (2016)
- [A Transformer-based Framework for Multivariate Time Series Representation Learning](https://mlsanity.com/paper/2010.02803/) (2020)
- [Generating Videos with Scene Dynamics](https://mlsanity.com/paper/1609.02612/) (2016)
- [Real-valued (Medical) Time Series Generation with Recurrent Conditional GANs](https://mlsanity.com/paper/1706.02633/) (2017)
- [Learning Generative Models with Sinkhorn Divergences](https://mlsanity.com/paper/1706.00292/) (2017)
- [The relativistic discriminator: a key element missing from standard GAN](https://mlsanity.com/paper/1807.00734/) (2018)
- [Professor Forcing: A New Algorithm for Training Recurrent Networks](https://mlsanity.com/paper/1610.09038/) (2016)
- [Generating Multi-label Discrete Patient Records using Generative Adversarial Networks](https://mlsanity.com/paper/1703.06490/) (2017)
- [C-RNN-GAN: Continuous recurrent neural networks with adversarial training](https://mlsanity.com/paper/1611.09904/) (2016)
- [Deep Transformer Models for Time Series Forecasting: The Influenza Prevalence Case](https://mlsanity.com/paper/2001.08317/) (2020)
- [COT-GAN: Generating Sequential Data via Causal Optimal Transport](https://mlsanity.com/paper/2006.08571/) (2020)
- [PPGAN: Privacy-preserving Generative Adversarial Network](https://mlsanity.com/paper/1910.02007/) (2019)
- [Adversarial Generation of Natural Language](https://mlsanity.com/paper/1705.10929/) (2017)
- [Dynamic Differential-Privacy Preserving SGD](https://mlsanity.com/paper/2111.00173/) (2021)
- [Fast, Accurate, and Simple Models for Tabular Data via Augmented Distillation](https://mlsanity.com/paper/2006.14284/) (2020)
- [Improving Neural Machine Translation with Conditional Sequence Generative Adversarial Nets](https://mlsanity.com/paper/1703.04887/) (2017)

## Built on by (most cited first, 10 of 10)

- [TSGBench: Time Series Generation Benchmark](https://mlsanity.com/paper/2309.03755/) (2023)
- [Constrained Posterior Sampling: Time Series Generation with Hard Constraints](https://mlsanity.com/paper/2410.12652/) (2024)
- [A Survey of Transformer Enabled Time Series Synthesis](https://mlsanity.com/paper/2406.02322/) (2024)
- [CTBench: Cryptocurrency Time Series Generation Benchmark](https://mlsanity.com/paper/2508.02758/) (2025)
- [Masked Multi-Step Multivariate Time Series Forecasting with Future Information](https://mlsanity.com/paper/2209.14413/) (2022)
- [Aircraft Trajectory Dataset Augmentation in Latent Space](https://mlsanity.com/paper/2506.07585/) (2025)
- [TransConv-DDPM: Enhanced Diffusion Model for Generating Time-Series Data in Healthcare](https://mlsanity.com/paper/2602.07033/) (2026)
- [Distribution estimation and change-point estimation for time series via DNN-based GANs](https://mlsanity.com/paper/2211.14577/) (2022)
- [Generative models for wearables data](https://mlsanity.com/paper/2307.16664/) (2023)
- [MGE: A Training-Free and Efficient Model Generation and Enhancement Scheme](https://mlsanity.com/paper/2402.17486/) (2024)

## Abstract (arXiv)

Many real-world tasks are plagued by limitations on data: in some instances very little data is available and in others, data is protected by privacy enforcing regulations (e.g. GDPR). We consider limitations posed specifically on time-series data and present a model that can generate synthetic time-series which can be used in place of real data. A model that generates synthetic time-series data has two objectives: 1) to capture the stepwise conditional distribution of real sequences, and 2) to faithfully model the joint distribution of entire real sequences. Autoregressive models trained via maximum likelihood estimation can be used in a system where previous predictions are fed back in and used to predict future ones; in such models, errors can accrue over time. Furthermore, a plausible initial value is required making MLE based models not really generative. Many downstream tasks learn to model conditional distributions of the time-series, hence, synthetic data drawn from a generative model must satisfy 1) in addition to performing 2). We present TsT-GAN, a framework that capitalises on the Transformer architecture to satisfy the desiderata and compare its performance against five state-of-the-art models on five datasets and show that TsT-GAN achieves higher predictive performance on all datasets.