mlsanityData to 14 Oct 2026

Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects

arXiv 2306.10125 ↗First posted 16 Jun 2023cs.LG cs.AI eess.SPcited by 99

Not on the map. Why?

Abstract (arXiv)

Self-supervised learning (SSL) has recently achieved impressive performance on various time series tasks. The most prominent advantage of SSL is that it reduces the dependence on labeled data. Based on the pre-training and fine-tuning strategy, even a small amount of labeled data can achieve high performance. Compared with many published self-supervised surveys on computer vision and natural language processing, a comprehensive survey for time series SSL is still missing. To fill this gap, we review current state-of-the-art SSL methods for time series data in this article. To this end, we first comprehensively review existing surveys related to SSL and time series, and then provide a new taxonomy of existing time series SSL methods by summarizing them from three perspectives: generative-based, contrastive-based, and adversarial-based. These methods are further divided into ten subcategories with detailed reviews and discussions about their key intuitions, main frameworks, advantages and disadvantages. To facilitate the experiments and validation of time series SSL methods, we also summarize datasets commonly used in time series forecasting, classification, anomaly detection, and clustering tasks. Finally, we present the future directions of SSL for time series analysis.

Key facts. Cited by 99 papers in the map (data through 14 Oct 2026). Builds on 146 of its 249 references that are in the map. First posted on arXiv 16 Jun 2023.

Builds on

146 of 249 references.

References · 146

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  6. 2023Segment Anythingcited by 8,234
  7. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  8. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  9. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  10. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  11. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  12. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
38 more
  1. 2013Intriguing properties of neural networkscited by 5,155
  2. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  3. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  4. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  5. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  6. 2020Supervised Contrastive Learningcited by 2,829
  7. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  8. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  9. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
  10. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
  11. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  12. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,476
  13. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
  14. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,318
  15. 2019Contrastive Multiview Codingcited by 1,301
  16. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,210
  17. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,114
  18. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,058
  19. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,043
  20. 2019An Introduction to Variational Autoencoderscited by 1,000
  21. 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 879
  22. 2020What Makes for Good Views for Contrastive Learning?cited by 846
  23. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 808
  24. 2016Universal adversarial perturbationscited by 784
  25. 2020Self-supervised Learning: Generative or Contrastivecited by 694
  26. 2021Maximum Likelihood Training of Score-Based Diffusion Modelscited by 672
  27. 2019Self-supervised Visual Feature Learning with Deep Neural Networks: A Surveycited by 657
  28. 2020Prototypical Contrastive Learning of Unsupervised Representationscited by 640
  29. 2021CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputationcited by 582
  30. 2018Deep learning for time series classification: a reviewcited by 576
  31. 2020Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networkscited by 566
  32. 2020A Survey on Contrastive Self-supervised Learningcited by 554
  33. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 505
  34. 2022Transformers in Time Series: A Surveycited by 497
  35. 2021TS2Vec: Towards Universal Representation of Time Seriescited by 471
  36. 2020Time Series Forecasting With Deep Learning: A Surveycited by 459
  37. 2020A Transformer-based Framework for Multivariate Time Series Representation Learningcited by 453
  38. 2018The UCR Time Series Archivecited by 431

50 most cited of 146 All 146 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 10; 2024: 34; 2025: 30; 2026: 25.

99 papers.

Sort:

By area:

  1. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 700
  2. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 614
  3. 2024Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Expertscited by 287
  4. 2023A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly Detectioncited by 145
  5. 2023Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlookcited by 110
  6. 2024VisionTS: Visual Masked Autoencoders Are Free-Lunch Zero-Shot Time Series Forecasterscited by 88
  7. 2024Time-MMD: Multi-Domain Multimodal Dataset for Time Series Analysiscited by 74
  8. 2024Position: What Can Large Language Models Tell Us about Time Series Analysiscited by 47
  9. 2024Towards a General Time Series Anomaly Detector with Adaptive Bottlenecks and Dual Adversarial Decoderscited by 38
  10. 2024TimeDiT: General-purpose Diffusion Transformers for Time Series Foundation Modelcited by 33
  11. 2024Towards Neural Scaling Laws for Time Series Foundation Modelscited by 33
  12. 2024Universal Time-Series Representation Learning: A Surveycited by 30
38 more
  1. 2024Large Language Model Guided Knowledge Distillation for Time Series Anomaly Detectioncited by 26
  2. 2024Time-FFM: Towards LM-Empowered Federated Foundation Model for Time Series Forecastingcited by 26
  3. 2023T-Rep: Representation Learning for Time Series using Time-Embeddingscited by 19
  4. 2023Correlation-aware Spatial-Temporal Graph Learning for Multivariate Time-series Anomaly Detectioncited by 18
  5. 2024On the Identification of Temporally Causal Representation with Instantaneous Dependencecited by 16
  6. 2025Towards a general-purpose foundation model for fMRI analysiscited by 15
  7. 2024TimeDART: A Diffusion Autoregressive Transformer for Self-Supervised Time Series Representationcited by 14
  8. 2023Unraveling the "Anomaly" in Time Series Anomaly Detection: A Self-supervised Tri-domain Solutioncited by 12
  9. 2023TimeDRL: Disentangled Representation Learning for Multivariate Time-Seriescited by 10
  10. 2025Learning Soft Sparse Shapes for Efficient Time-Series Classificationcited by 10
  11. 2024Self-Supervised Representation Learning for Geospatial Objects: A Surveycited by 9
  12. 2025Vision-Enhanced Time Series Forecasting via Latent Diffusion Modelscited by 9
  13. 2026Position: Beyond Model-Centric Prediction -- Agentic Time Series Forecastingcited by 9
  14. 2023Prospective Role of Foundation Models in Advancing Autonomous Vehiclescited by 7
  15. 2025Generalizable and Explainable Deep Learning for Medical Image Computing: An Overviewcited by 6
  16. 2024Denoising-Aware Contrastive Learning for Noisy Time Seriescited by 5
  17. 2024Time Series Analysis for Education: Methods, Applications, and Future Directionscited by 5
  18. 2024EMIT- Event-Based Masked Auto Encoding for Irregular Time Seriescited by 5
  19. 2025Exploring Structural Degradation in Dense Representations for Self-supervised Learningcited by 5
  20. 2024Towards Foundation Time Series Model: To Synthesize Or Not To Synthesize?cited by 4
  21. 2024From Orthogonality to Dependency: Learning Disentangled Representation for Multi-Modal Time-Series Sensing Signalscited by 4
  22. 2025Homophily Heterogeneity Matters in Graph Federated Learning: A Spectrum Sharing and Complementing Perspectivecited by 4
  23. 2025Fuzzy Cluster-Aware Contrastive Clustering for Time Seriescited by 4
  24. 2025Multi-View Contrastive Learning for Robust Domain Adaptation in Medical Time Series Analysiscited by 4
  25. 2023Self-Distilled Representation Learning for Time Seriescited by 3
  26. 2024Toward Physics-guided Time Series Embeddingcited by 3
  27. 2024A Survey of Spatio-Temporal EEG data Analysis: from Models to Applicationscited by 3
  28. 2024DisenTS: Disentangled Channel Evolving Pattern Modeling for Multivariate Time Series Forecastingcited by 3
  29. 2024Weakly Augmented Variational Autoencoder in Time Series Anomaly Detectioncited by 2
  30. 2024EEG-MACS: Manifold Attention and Confidence Stratification for EEG-based Cross-Center Brain Disease Diagnosis under Unreliable Annotationscited by 2
  31. 2024Semi-supervised Anomaly Detection via Adaptive Reinforcement Learning-Enabled Method with Causal Inference for Sensor Signalscited by 2
  32. 2024Self-Supervised Learning of Disentangled Representations for Multivariate Time-Seriescited by 2
  33. 2024Recurrent Neural Goodness-of-Fit Test for Time Seriescited by 2
  34. 2025Representation Learning of Limit Order Book: A Comprehensive Study and Benchmarkingcited by 2
  35. 2026Bi-level Heterogeneous Learning for Time Series Foundation Models: A Federated Learning Approachcited by 2
  36. 2026Beyond Patient Invariance: Learning Cardiac Dynamics via Action-Conditioned JEPAscited by 2
  37. 2026A General Framework for Generative Self-supervised Learning in Non-invasive Estimation of Physiological Parameters Using Photoplethysmographycited by 2
  38. 2024Rank Supervised Contrastive Learning for Time Series Classificationcited by 1

50 most cited of 99 All 99 (JSON)