mlsanityData to 14 Oct 2026

Universal Time-Series Representation Learning: A Survey

arXiv 2401.03717 ↗First posted 8 Jan 2024cs.LG cs.AIcited by 30

Not on the map. Why?

Abstract (arXiv)

Time-series data exists in every corner of real-world systems and services, ranging from satellites in the sky to wearable devices on human bodies. Learning representations by extracting and inferring valuable information from these time series is crucial for understanding the complex dynamics of particular phenomena and enabling informed decisions. With the learned representations, we can perform numerous downstream analyses more effectively. Among several approaches, deep learning has demonstrated remarkable performance in extracting hidden patterns and features from time-series data without manual feature engineering. This survey first presents a novel taxonomy based on three fundamental elements in designing state-of-the-art universal representation learning methods for time series. According to the proposed taxonomy, we comprehensively review existing studies and discuss their intuitions and insights into how these methods enhance the quality of learned representations. Finally, as a guideline for future studies, we summarize commonly used experimental setups and datasets and discuss several promising research directions. An up-to-date corresponding resource is available at https://github.com/itouchz/awesome-deep-time-series-representations.

Key facts. Cited by 30 papers in the map (data through 14 Oct 2026). Builds on 142 of its 264 references that are in the map. First posted on arXiv 8 Jan 2024.

Builds on

142 of 264 references.

References · 142

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  4. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  5. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  6. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  7. 2018Neural Ordinary Differential Equationscited by 3,779
  8. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
  9. 2017The Kinetics Human Action Video Datasetcited by 1,909
  10. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,476
  11. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,058
  12. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 950
38 more
  1. 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 879
  2. 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 739
  3. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 632
  4. 2018Real-world Anomaly Detection in Surveillance Videoscited by 577
  5. 2018Deep learning for time series classification: a reviewcited by 576
  6. 2016Recurrent Neural Networks for Multivariate Time Series with Missing Valuescited by 554
  7. 2020Rescaling Egocentric Visioncited by 540
  8. 2022Transformers in Time Series: A Surveycited by 497
  9. 2021TS2Vec: Towards Universal Representation of Time Seriescited by 471
  10. 2020Time Series Forecasting With Deep Learning: A Surveycited by 459
  11. 2020A Transformer-based Framework for Multivariate Time Series Representation Learningcited by 453
  12. 2018The UCR Time Series Archivecited by 431
  13. 2018Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholdingcited by 410
  14. 2020Spatiotemporal Contrastive Video Representation Learningcited by 372
  15. 2021Time-Series Representation Learning via Temporal and Contextual Contrastingcited by 345
  16. 2018The UEA multivariate time series classification archive, 2018cited by 338
  17. 2019Unsupervised Scalable Representation Learning for Multivariate Time Seriescited by 269
  18. 2021Monash Time Series Forecasting Archivecited by 267
  19. 2018The Sound of Pixelscited by 259
  20. 2022Self-Supervised Contrastive Pre-Training For Time Series via Time-Frequency Consistencycited by 250
  21. 2021Multi-Time Attention Networks for Irregularly Sampled Time Seriescited by 238
  22. 2020Time Series Data Augmentation for Deep Learning: A Surveycited by 226
  23. 2018Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applicationscited by 224
  24. 2021Unsupervised Representation Learning for Time Series with Temporal Neighborhood Codingcited by 224
  25. 2022Learning Video Representations from Large Language Modelscited by 205
  26. 2023FITS: Modeling Time Series with 10k Parameterscited by 195
  27. 2019Latent ODEs for Irregularly-Sampled Time Seriescited by 183
  28. 2023TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Seriescited by 169
  29. 2020Current Time Series Anomaly Detection Benchmarks are Flawed and are Creating the Illusion of Progresscited by 151
  30. 2021Self-Supervised Representation Learning: Introduction, Advances and Challengescited by 148
  31. 2023A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly Detectioncited by 145
  32. 2020Self-supervised Video Representation Learning by Pace Predictioncited by 139
  33. 2023Neural Architecture Search: Insights from 1000 Paperscited by 139
  34. 2020Memory-augmented Dense Predictive Coding for Video Representation Learningcited by 137
  35. 2021TCLR: Temporal Contrastive Learning for Video Representationcited by 136
  36. 2022Deep Learning for Time Series Anomaly Detection: A Surveycited by 131
  37. 2020An Empirical Survey of Data Augmentation for Time Series Classification with Neural Networkscited by 128
  38. 2020Deep Learning for Sensor-based Human Activity Recognition: Overview, Challenges and Opportunitiescited by 126

50 most cited of 142 All 142 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 7; 2025: 13; 2026: 10.

30 papers.

Sort:

By area:

  1. 2024UniTS: A Unified Multi-Task Time Series Modelcited by 94
  2. 2024TimeMachine: A Time Series is Worth 4 Mambas for Long-term Forecastingcited by 64
  3. 2024Empowering Time Series Analysis with Foundation Models: A Comprehensive Surveycited by 23
  4. 2024Frequency-Masked Embedding Inference: A Non-Contrastive Approach for Time Series Representation Learningcited by 9
  5. 2025mTSBench: Benchmarking Multivariate Time Series Anomaly Detection and Model Selection at Scalecited by 7
  6. 2024UniCL: A Universal Contrastive Learning Framework for Large Time Series Modelscited by 5
  7. 2026Can we generate portable representations for clinical time series data using LLMs?cited by 4
  8. 2025VarDrop: Enhancing Training Efficiency by Reducing Variate Redundancy in Periodic Time Series Forecastingcited by 3
  9. 2026Channel-wise Retrieval for Multivariate Time Series Forecastingcited by 3
  10. 2024Nonlinear time-series embedding by monotone variational inequalitycited by 2
  11. 2024Channel-aware Contrastive Conditional Diffusion for Multivariate Probabilistic Time Series Forecastingcited by 2
  12. 2025Representation Learning of Limit Order Book: A Comprehensive Study and Benchmarkingcited by 2
18 more
  1. 2025MONAQ: Multi-Objective Neural Architecture Querying for Time-Series Analysis on Resource-Constrained Devicescited by 2
  2. 2025Structure-preserving contrastive learning for spatial time seriescited by 1
  3. 2025Efficient Hierarchical Contrastive Self-supervising Learning for Time Series Classification via Importance-aware Resolution Selectioncited by 1
  4. 2025Universal Domain Adaptation Benchmark for Time Series Data Representationcited by 1
  5. 2025Detecting Multilevel Manipulation from Limit Order Book via Cascaded Contrastive Representation Learningcited by 1
  6. 2026XCTFormer: Leveraging Cross-Channel and Cross-Time Dependencies for Enhanced Time-Series Analysiscited by 1
  7. 2026Self-Supervised Online Robot-Agnostic Traversability Estimation for Open-World Environmentscited by 1
  8. 2025PLanTS: Periodicity-aware Latent-state Representation Learning for Multivariate Time Seriescited by 0
  9. 2025Forecasting-based Biomedical Time-series Data Synthesis for Open Data and Robust AIcited by 0
  10. 2025Weighted Contrastive Learning for Anomaly-Aware Time-Series Forecastingcited by 0
  11. 2025Symplectic Representation of Legendre Dynamicscited by 0
  12. 2025Enhancing Zero-Shot Time Series Forecasting in Off-the-Shelf LLMs via Noise Injectioncited by 0
  13. 2026Deep Distance Measurement Method for Unsupervised Multivariate Time Series Similarity Retrievalcited by 0
  14. 2026When Does Self-Supervised Learning Transfer to Time-Series Tasks?cited by 0
  15. 2026ADAPTOOD: Uncertainty-Aware Fine-Tuning for Out-of-Distribution ECG Time Series Modelscited by 0
  16. 2026Traceable Multi-Agent System for Knowledge-Based Forecastingcited by 0
  17. 2026Time series generation with spectrally aligned latent flow matchingcited by 0
  18. 2026CT-Miner: Fast and Coarse-Grained Time-Series Pattern Mining via Cartesian Treescited by 0