mlsanityData to 7 Oct 2026

TS2Vec: Towards Universal Representation of Time Series

arXiv 2106.10466 ↗First posted 19 Jun 2021cs.LG cs.AIcited by 463

Not on the map. Why?

Abstract (arXiv)

This paper presents TS2Vec, a universal framework for learning representations of time series in an arbitrary semantic level. Unlike existing methods, TS2Vec performs contrastive learning in a hierarchical way over augmented context views, which enables a robust contextual representation for each timestamp. Furthermore, to obtain the representation of an arbitrary sub-sequence in the time series, we can apply a simple aggregation over the representations of corresponding timestamps. We conduct extensive experiments on time series classification tasks to evaluate the quality of time series representations. As a result, TS2Vec achieves significant improvement over existing SOTAs of unsupervised time series representation on 125 UCR datasets and 29 UEA datasets. The learned timestamp-level representations also achieve superior results in time series forecasting and anomaly detection tasks. A linear regression trained on top of the learned representations outperforms previous SOTAs of time series forecasting. Furthermore, we present a simple way to apply the learned representations for unsupervised anomaly detection, which establishes SOTA results in the literature. The source code is publicly available at https://github.com/yuezhihan/ts2vec.

Key facts. Cited by 463 papers in the map (data through 7 Oct 2026). Builds on 27 of its 52 references that are in the map. First posted on arXiv 19 Jun 2021.

Builds on

27 of 52 references.

References · 27

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  3. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  4. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,141
  5. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,447
  6. 2017DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networkscited by 1,004
  7. 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 869
  8. 2019N-BEATS: Neural basis expansion analysis for interpretable time series forecastingcited by 744
  9. 2019Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecastingcited by 606
  10. 2020Dense Contrastive Learning for Self-Supervised Visual Pre-Trainingcited by 512
  11. 2020A Transformer-based Framework for Multivariate Time Series Representation Learningcited by 449
  12. 2018The UCR Time Series Archivecited by 421
15 more
  1. 2018The UEA multivariate time series classification archive, 2018cited by 330
  2. 2021Spectral Temporal Graph Neural Network for Multivariate Time-series Forecastingcited by 298
  3. 2019Unsupervised Scalable Representation Learning for Multivariate Time Seriescited by 267
  4. 2018Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applicationscited by 223
  5. 2021Unsupervised Representation Learning for Time Series with Temporal Neighborhood Codingcited by 223
  6. 2017A Multi-Horizon Quantile Recurrent Forecastercited by 211
  7. 2018An efficient framework for learning sentence representationscited by 197
  8. 2020How Much Position Information Do Convolutional Neural Networks Encode?cited by 171
  9. 2019Time-Series Anomaly Detection Service at Microsoftcited by 163
  10. 2020Unsupervised Learning of Dense Visual Representationscited by 147
  11. 2020On Translation Invariance in CNNs: Convolutional Layers can Exploit Absolute Spatial Locationcited by 141
  12. 2020Self-training and Pre-training are Complementary for Speech Recognitioncited by 98
  13. 2017TimeNet: Pre-trained deep recurrent neural network for time series classificationcited by 46
  14. 2018Random Warping Series: A Random Features Method for Time-Series Embeddingcited by 12
  15. 2020Seed the Views: Hierarchical Semantic Alignment for Contrastive Representation Learningcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 20; 2023: 92; 2024: 128; 2025: 114; 2026: 107.

463 papers.

Sort:

By area:

  1. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
  2. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 588
  3. 2024MOMENT: A Family of Open Time-series Foundation Modelscited by 453
  4. 2024Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Expertscited by 280
  5. 2022Self-Supervised Contrastive Pre-Training For Time Series via Time-Frequency Consistencycited by 246
  6. 2024Foundation Models for Time Series Analysis: A Tutorial and Surveycited by 234
  7. 2024Timer: Generative Pre-trained Transformers Are Large Time Series Modelscited by 198
  8. 2024Deep Time Series Models: A Comprehensive Survey and Benchmarkcited by 191
  9. 2023TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Seriescited by 165
  10. 2022CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecastingcited by 143
  11. 2023TSMixer: Lightweight MLP-Mixer Model for Multivariate Time Series Forecastingcited by 136
  12. 2022Deep Learning for Time Series Anomaly Detection: A Surveycited by 130
38 more
  1. 2023LLM4TS: Aligning Pre-Trained LLMs as Data-Efficient Time-Series Forecasterscited by 127
  2. 2024Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Seriescited by 120
  3. 2023SimMTM: A Simple Pre-Training Framework for Masked Time-Series Modelingcited by 112
  4. 2023Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlookcited by 109
  5. 2023Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospectscited by 97
  6. 2022Unsupervised Time-Series Representation Learning with Iterative Bilinear Temporal-Spectral Fusioncited by 96
  7. 2022Deep Isolation Forest for Anomaly Detectioncited by 90
  8. 2024VisionTS: Visual Masked Autoencoders Are Free-Lunch Zero-Shot Time Series Forecasterscited by 87
  9. 2024TSLANet: Rethinking Transformers for Time Series Representation Learningcited by 71
  10. 2023Deep Learning for Time Series Classification and Extrinsic Regression: A Current Surveycited by 68
  11. 2023Ti-MAE: Self-Supervised Masked Time Series Autoencoderscited by 63
  12. 2024TimeMachine: A Time Series is Worth 4 Mambas for Long-term Forecastingcited by 63
  13. 2024ContiFormer: Continuous-Time Transformer for Irregular Time Series Modelingcited by 61
  14. 2023Contrast Everything: A Hierarchical Contrastive Framework for Medical Time-Seriescited by 59
  15. 2024Medformer: A Multi-Granularity Patching Transformer for Medical Time-Series Classificationcited by 53
  16. 2023Time Series Contrastive Learning with Information-Aware Augmentationscited by 52
  17. 2023A Survey on Time-Series Pre-Trained Modelscited by 52
  18. 2023TimesURL: Self-supervised Contrastive Learning for Universal Time Series Representation Learningcited by 51
  19. 2024EEGFormer: Towards Transferable and Interpretable Large-Scale EEG Foundation Modelcited by 51
  20. 2023MSGNet: Learning Multi-Scale Inter-Series Correlations for Multivariate Time Series Forecastingcited by 50
  21. 2023DSformer: A Double Sampling Transformer for Multivariate Time Series Long-term Predictioncited by 49
  22. 2023Soft Contrastive Learning for Time Seriescited by 49
  23. 2022Self-Supervised Time Series Representation Learning via Cross Reconstruction Transformercited by 47
  24. 2022Learning Fast and Slow for Online Time Series Forecastingcited by 43
  25. 2023TimeMAE: Self-Supervised Representations of Time Series with Decoupled Masked Autoencoderscited by 42
  26. 2023BasisFormer: Attention-based Time Series Forecasting with Learnable and Interpretable Basiscited by 41
  27. 2022Self-supervised Contrastive Representation Learning for Semi-supervised Time-Series Classificationcited by 39
  28. 2024TOTEM: TOkenized Time Series EMbeddings for General Time Series Analysiscited by 37
  29. 2024Towards a General Time Series Anomaly Detector with Adaptive Bottlenecks and Dual Adversarial Decoderscited by 37
  30. 2022Contrastive Learning for Unsupervised Domain Adaptation of Time Seriescited by 36
  31. 2025Pulse-PPG: An Open-Source Field-Trained PPG Foundation Model for Wearable Applications Across Lab and Field Settingscited by 35
  32. 2025ITFormer: Bridging Time Series and Natural Language for Multi-Modal QA with Large-Scale Multitask Datasetcited by 35
  33. 2023Learning to Embed Time Series Patches Independentlycited by 34
  34. 2024TimeDiT: General-purpose Diffusion Transformers for Time Series Foundation Modelcited by 33
  35. 2024A Survey of Deep Learning and Foundation Models for Time Series Forecastingcited by 32
  36. 2023FOCAL: Contrastive Learning for Multimodal Time-Series Sensing Signals in Factorized Orthogonal Latent Spacecited by 31
  37. 2024Root Cause Analysis In Microservice Using Neural Granger Causal Discoverycited by 31
  38. 2024EasyTime: Time Series Forecasting Made Easycited by 30

50 most cited of 463 All 463 (JSON)