# Universal Time-Series Representation Learning: A Survey

arXiv: https://arxiv.org/abs/2401.03717 · HTML: https://mlsanity.com/paper/2401.03717/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 142 of its 264 references that are in the map.
- First posted on arXiv 8 Jan 2024.

## Builds on (most cited first, 30 of 142)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis](https://mlsanity.com/paper/2210.02186/) (2022)
- [HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips](https://mlsanity.com/paper/1906.03327/) (2019)
- [Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks](https://mlsanity.com/paper/1703.07015/) (2017)
- [Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution](https://mlsanity.com/paper/2202.10054/) (2022)
- [One Fits All:Power General Time Series Analysis by Pretrained LM](https://mlsanity.com/paper/2302.11939/) (2023)
- [Real-world Anomaly Detection in Surveillance Videos](https://mlsanity.com/paper/1801.04264/) (2018)
- [Deep learning for time series classification: a review](https://mlsanity.com/paper/1809.04356/) (2018)
- [Recurrent Neural Networks for Multivariate Time Series with Missing Values](https://mlsanity.com/paper/1606.01865/) (2016)
- [Rescaling Egocentric Vision](https://mlsanity.com/paper/2006.13256/) (2020)
- [Transformers in Time Series: A Survey](https://mlsanity.com/paper/2202.07125/) (2022)
- [TS2Vec: Towards Universal Representation of Time Series](https://mlsanity.com/paper/2106.10466/) (2021)
- [Time Series Forecasting With Deep Learning: A Survey](https://mlsanity.com/paper/2004.13408/) (2020)
- [A Transformer-based Framework for Multivariate Time Series Representation Learning](https://mlsanity.com/paper/2010.02803/) (2020)
- [The UCR Time Series Archive](https://mlsanity.com/paper/1810.07758/) (2018)
- [Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding](https://mlsanity.com/paper/1802.04431/) (2018)
- [Spatiotemporal Contrastive Video Representation Learning](https://mlsanity.com/paper/2008.03800/) (2020)
- [Time-Series Representation Learning via Temporal and Contextual Contrasting](https://mlsanity.com/paper/2106.14112/) (2021)
- [The UEA multivariate time series classification archive, 2018](https://mlsanity.com/paper/1811.00075/) (2018)
- [Unsupervised Scalable Representation Learning for Multivariate Time Series](https://mlsanity.com/paper/1901.10738/) (2019)
- [Monash Time Series Forecasting Archive](https://mlsanity.com/paper/2105.06643/) (2021)

## Built on by (most cited first, 30 of 30)

- [UniTS: A Unified Multi-Task Time Series Model](https://mlsanity.com/paper/2403.00131/) (2024)
- [TimeMachine: A Time Series is Worth 4 Mambas for Long-term Forecasting](https://mlsanity.com/paper/2403.09898/) (2024)
- [Empowering Time Series Analysis with Foundation Models: A Comprehensive Survey](https://mlsanity.com/paper/2405.02358/) (2024)
- [Frequency-Masked Embedding Inference: A Non-Contrastive Approach for Time Series Representation Learning](https://mlsanity.com/paper/2412.20790/) (2024)
- [mTSBench: Benchmarking Multivariate Time Series Anomaly Detection and Model Selection at Scale](https://mlsanity.com/paper/2506.21550/) (2025)
- [UniCL: A Universal Contrastive Learning Framework for Large Time Series Models](https://mlsanity.com/paper/2405.10597/) (2024)
- [Can we generate portable representations for clinical time series data using LLMs?](https://mlsanity.com/paper/2603.23987/) (2026)
- [VarDrop: Enhancing Training Efficiency by Reducing Variate Redundancy in Periodic Time Series Forecasting](https://mlsanity.com/paper/2501.14183/) (2025)
- [Channel-wise Retrieval for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2604.05543/) (2026)
- [Nonlinear time-series embedding by monotone variational inequality](https://mlsanity.com/paper/2406.06894/) (2024)
- [Channel-aware Contrastive Conditional Diffusion for Multivariate Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2410.02168/) (2024)
- [Representation Learning of Limit Order Book: A Comprehensive Study and Benchmarking](https://mlsanity.com/paper/2505.02139/) (2025)
- [MONAQ: Multi-Objective Neural Architecture Querying for Time-Series Analysis on Resource-Constrained Devices](https://mlsanity.com/paper/2505.10607/) (2025)
- [Structure-preserving contrastive learning for spatial time series](https://mlsanity.com/paper/2502.06380/) (2025)
- [Efficient Hierarchical Contrastive Self-supervising Learning for Time Series Classification via Importance-aware Resolution Selection](https://mlsanity.com/paper/2502.10567/) (2025)
- [Universal Domain Adaptation Benchmark for Time Series Data Representation](https://mlsanity.com/paper/2505.17899/) (2025)
- [Detecting Multilevel Manipulation from Limit Order Book via Cascaded Contrastive Representation Learning](https://mlsanity.com/paper/2508.17086/) (2025)
- [XCTFormer: Leveraging Cross-Channel and Cross-Time Dependencies for Enhanced Time-Series Analysis](https://mlsanity.com/paper/2605.18534/) (2026)
- [Self-Supervised Online Robot-Agnostic Traversability Estimation for Open-World Environments](https://mlsanity.com/paper/2605.28442/) (2026)
- [PLanTS: Periodicity-aware Latent-state Representation Learning for Multivariate Time Series](https://mlsanity.com/paper/2509.05478/) (2025)
- [Forecasting-based Biomedical Time-series Data Synthesis for Open Data and Robust AI](https://mlsanity.com/paper/2510.04622/) (2025)
- [Weighted Contrastive Learning for Anomaly-Aware Time-Series Forecasting](https://mlsanity.com/paper/2512.07569/) (2025)
- [Symplectic Representation of Legendre Dynamics](https://mlsanity.com/paper/2512.19409/) (2025)
- [Enhancing Zero-Shot Time Series Forecasting in Off-the-Shelf LLMs via Noise Injection](https://mlsanity.com/paper/2512.20140/) (2025)
- [Deep Distance Measurement Method for Unsupervised Multivariate Time Series Similarity Retrieval](https://mlsanity.com/paper/2603.12544/) (2026)
- [When Does Self-Supervised Learning Transfer to Time-Series Tasks?](https://mlsanity.com/paper/2605.19462/) (2026)
- [ADAPTOOD: Uncertainty-Aware Fine-Tuning for Out-of-Distribution ECG Time Series Models](https://mlsanity.com/paper/2606.04164/) (2026)
- [Traceable Multi-Agent System for Knowledge-Based Forecasting](https://mlsanity.com/paper/2608.03339/) (2026)
- [Time series generation with spectrally aligned latent flow matching](https://mlsanity.com/paper/2609.21989/) (2026)
- [CT-Miner: Fast and Coarse-Grained Time-Series Pattern Mining via Cartesian Trees](https://mlsanity.com/paper/2610.05330/) (2026)

## Abstract (arXiv)

Time-series data exists in every corner of real-world systems and services, ranging from satellites in the sky to wearable devices on human bodies. Learning representations by extracting and inferring valuable information from these time series is crucial for understanding the complex dynamics of particular phenomena and enabling informed decisions. With the learned representations, we can perform numerous downstream analyses more effectively. Among several approaches, deep learning has demonstrated remarkable performance in extracting hidden patterns and features from time-series data without manual feature engineering. This survey first presents a novel taxonomy based on three fundamental elements in designing state-of-the-art universal representation learning methods for time series. According to the proposed taxonomy, we comprehensively review existing studies and discuss their intuitions and insights into how these methods enhance the quality of learned representations. Finally, as a guideline for future studies, we summarize commonly used experimental setups and datasets and discuss several promising research directions. An up-to-date corresponding resource is available at https://github.com/itouchz/awesome-deep-time-series-representations.