# Time Series Generation with Masked Autoencoder

arXiv: https://arxiv.org/abs/2201.07006 · HTML: https://mlsanity.com/paper/2201.07006/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 35 references that are in the map.
- First posted on arXiv 14 Jan 2022.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [N-BEATS: Neural basis expansion analysis for interpretable time series forecasting](https://mlsanity.com/paper/1905.10437/) (2019)
- [Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting](https://mlsanity.com/paper/1907.00235/) (2019)
- [Real-valued (Medical) Time Series Generation with Recurrent Conditional GANs](https://mlsanity.com/paper/1706.02633/) (2017)
- [BRITS: Bidirectional Recurrent Imputation for Time Series](https://mlsanity.com/paper/1805.10572/) (2018)
- [C-RNN-GAN: Continuous recurrent neural networks with adversarial training](https://mlsanity.com/paper/1611.09904/) (2016)
- [Stabilizing Generative Adversarial Networks: A Survey](https://mlsanity.com/paper/1910.00927/) (2019)
- [Time Series Data Imputation: A Survey on Deep Learning Approaches](https://mlsanity.com/paper/2011.11347/) (2020)
- [Decision-Aware Conditional GANs for Time Series Data](https://mlsanity.com/paper/2009.12682/) (2020)
- [Autoencoders as Weight Initialization of Deep Classification Networks for Cancer versus Cancer Studies](https://mlsanity.com/paper/2001.05253/) (2020)

## Built on by (most cited first, 12 of 12)

- [Ti-MAE: Self-Supervised Masked Time Series Autoencoders](https://mlsanity.com/paper/2301.08871/) (2023)
- [A Survey on Masked Autoencoder for Self-supervised Learning in Vision and Beyond](https://mlsanity.com/paper/2208.00173/) (2022)
- [Vector Quantized Time Series Generation with a Bidirectional Prior Model](https://mlsanity.com/paper/2303.04743/) (2023)
- [A Survey of Deep Learning and Foundation Models for Time Series Forecasting](https://mlsanity.com/paper/2401.13912/) (2024)
- [Data Augmentation for Time-Series Classification: An Extensive Empirical Study and Comprehensive Survey](https://mlsanity.com/paper/2310.10060/) (2023)
- [Aerial Image Object Detection With Vision Transformer Detector (ViTDet)](https://mlsanity.com/paper/2301.12058/) (2023)
- [Masked Multi-Step Multivariate Time Series Forecasting with Future Information](https://mlsanity.com/paper/2209.14413/) (2022)
- [PT-Tuning: Bridging the Gap between Time Series Masked Reconstruction and Forecasting via Prompt Token Tuning](https://mlsanity.com/paper/2311.03768/) (2023)
- [An Energy-Efficient Ensemble Approach for Mitigating Data Incompleteness in IoT Applications](https://mlsanity.com/paper/2403.10371/) (2024)
- [One Masked Model is All You Need for Sensor Fault Detection, Isolation and Accommodation](https://mlsanity.com/paper/2403.16153/) (2024)
- [Market-Derived Financial Sentiment Analysis: Context-Aware Language Models for Crypto Forecasting](https://mlsanity.com/paper/2502.14897/) (2025)
- [A Unified Contrastive-Generative Framework for Time Series Classification](https://mlsanity.com/paper/2508.09451/) (2025)

## Abstract (arXiv)

This paper shows that masked autoencoder with extrapolator (ExtraMAE) is a scalable self-supervised model for time series generation. ExtraMAE randomly masks some patches of the original time series and learns temporal dynamics by recovering the masked patches. Our approach has two core designs. First, ExtraMAE is self-supervised. Supervision allows ExtraMAE to effectively and efficiently capture the temporal dynamics of the original time series. Second, ExtraMAE proposes an extrapolator to disentangle two jobs of the decoder: recovering latent representations and mapping them back into the feature space. These unique designs enable ExtraMAE to consistently and significantly outperform state-of-the-art (SoTA) benchmarks in time series generation. The lightweight architecture also makes ExtraMAE fast and scalable. ExtraMAE shows outstanding behavior in various downstream tasks such as time series classification, prediction, and imputation. As a self-supervised generative model, ExtraMAE allows explicit management of the synthetic data. We hope this paper will usher in a new era of time series generation with self-supervised models.