# Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects

arXiv: https://arxiv.org/abs/2306.10125 · HTML: https://mlsanity.com/paper/2306.10125/

## Key facts

- Cited by 99 papers in the map (data through 14 Oct 2026).
- Builds on 146 of its 249 references that are in the map.
- First posted on arXiv 16 Jun 2023.

## Builds on (most cited first, 30 of 146)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere](https://mlsanity.com/paper/2005.10242/) (2020)
- [DeepFool: a simple and accurate method to fool deep neural networks](https://mlsanity.com/paper/1511.04599/) (2015)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis](https://mlsanity.com/paper/2210.02186/) (2022)

## Built on by (most cited first, 30 of 99)

- [Time-LLM: Time Series Forecasting by Reprogramming Large Language Models](https://mlsanity.com/paper/2310.01728/) (2023)
- [Unified Training of Universal Time Series Forecasting Transformers](https://mlsanity.com/paper/2402.02592/) (2024)
- [Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts](https://mlsanity.com/paper/2409.16040/) (2024)
- [A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly Detection](https://mlsanity.com/paper/2307.03759/) (2023)
- [Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook](https://mlsanity.com/paper/2310.10196/) (2023)
- [VisionTS: Visual Masked Autoencoders Are Free-Lunch Zero-Shot Time Series Forecasters](https://mlsanity.com/paper/2408.17253/) (2024)
- [Time-MMD: Multi-Domain Multimodal Dataset for Time Series Analysis](https://mlsanity.com/paper/2406.08627/) (2024)
- [Position: What Can Large Language Models Tell Us about Time Series Analysis](https://mlsanity.com/paper/2402.02713/) (2024)
- [Towards a General Time Series Anomaly Detector with Adaptive Bottlenecks and Dual Adversarial Decoders](https://mlsanity.com/paper/2405.15273/) (2024)
- [TimeDiT: General-purpose Diffusion Transformers for Time Series Foundation Model](https://mlsanity.com/paper/2409.02322/) (2024)
- [Towards Neural Scaling Laws for Time Series Foundation Models](https://mlsanity.com/paper/2410.12360/) (2024)
- [Universal Time-Series Representation Learning: A Survey](https://mlsanity.com/paper/2401.03717/) (2024)
- [Large Language Model Guided Knowledge Distillation for Time Series Anomaly Detection](https://mlsanity.com/paper/2401.15123/) (2024)
- [Time-FFM: Towards LM-Empowered Federated Foundation Model for Time Series Forecasting](https://mlsanity.com/paper/2405.14252/) (2024)
- [T-Rep: Representation Learning for Time Series using Time-Embeddings](https://mlsanity.com/paper/2310.04486/) (2023)
- [Correlation-aware Spatial-Temporal Graph Learning for Multivariate Time-series Anomaly Detection](https://mlsanity.com/paper/2307.08390/) (2023)
- [On the Identification of Temporally Causal Representation with Instantaneous Dependence](https://mlsanity.com/paper/2405.15325/) (2024)
- [Towards a general-purpose foundation model for fMRI analysis](https://mlsanity.com/paper/2506.11167/) (2025)
- [TimeDART: A Diffusion Autoregressive Transformer for Self-Supervised Time Series Representation](https://mlsanity.com/paper/2410.05711/) (2024)
- [Unraveling the "Anomaly" in Time Series Anomaly Detection: A Self-supervised Tri-domain Solution](https://mlsanity.com/paper/2311.11235/) (2023)
- [TimeDRL: Disentangled Representation Learning for Multivariate Time-Series](https://mlsanity.com/paper/2312.04142/) (2023)
- [Learning Soft Sparse Shapes for Efficient Time-Series Classification](https://mlsanity.com/paper/2505.06892/) (2025)
- [Self-Supervised Representation Learning for Geospatial Objects: A Survey](https://mlsanity.com/paper/2408.12133/) (2024)
- [Vision-Enhanced Time Series Forecasting via Latent Diffusion Models](https://mlsanity.com/paper/2502.14887/) (2025)
- [Position: Beyond Model-Centric Prediction -- Agentic Time Series Forecasting](https://mlsanity.com/paper/2602.01776/) (2026)
- [Prospective Role of Foundation Models in Advancing Autonomous Vehicles](https://mlsanity.com/paper/2405.02288/) (2023)
- [Generalizable and Explainable Deep Learning for Medical Image Computing: An Overview](https://mlsanity.com/paper/2503.08420/) (2025)
- [Denoising-Aware Contrastive Learning for Noisy Time Series](https://mlsanity.com/paper/2406.04627/) (2024)
- [Time Series Analysis for Education: Methods, Applications, and Future Directions](https://mlsanity.com/paper/2408.13960/) (2024)
- [EMIT- Event-Based Masked Auto Encoding for Irregular Time Series](https://mlsanity.com/paper/2409.16554/) (2024)

## Abstract (arXiv)

Self-supervised learning (SSL) has recently achieved impressive performance on various time series tasks. The most prominent advantage of SSL is that it reduces the dependence on labeled data. Based on the pre-training and fine-tuning strategy, even a small amount of labeled data can achieve high performance. Compared with many published self-supervised surveys on computer vision and natural language processing, a comprehensive survey for time series SSL is still missing. To fill this gap, we review current state-of-the-art SSL methods for time series data in this article. To this end, we first comprehensively review existing surveys related to SSL and time series, and then provide a new taxonomy of existing time series SSL methods by summarizing them from three perspectives: generative-based, contrastive-based, and adversarial-based. These methods are further divided into ten subcategories with detailed reviews and discussions about their key intuitions, main frameworks, advantages and disadvantages. To facilitate the experiments and validation of time series SSL methods, we also summarize datasets commonly used in time series forecasting, classification, anomaly detection, and clustering tasks. Finally, we present the future directions of SSL for time series analysis.