# T-Rep: Representation Learning for Time Series using Time-Embeddings

arXiv: https://arxiv.org/abs/2310.04486 · HTML: https://mlsanity.com/paper/2310.04486/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 54 references that are in the map.
- First posted on arXiv 6 Oct 2023.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [Transformers in Time Series: A Survey](https://mlsanity.com/paper/2202.07125/) (2022)
- [TS2Vec: Towards Universal Representation of Time Series](https://mlsanity.com/paper/2106.10466/) (2021)
- [A Transformer-based Framework for Multivariate Time Series Representation Learning](https://mlsanity.com/paper/2010.02803/) (2020)
- [The UCR Time Series Archive](https://mlsanity.com/paper/1810.07758/) (2018)
- [Time-Series Representation Learning via Temporal and Contextual Contrasting](https://mlsanity.com/paper/2106.14112/) (2021)
- [Unsupervised Scalable Representation Learning for Multivariate Time Series](https://mlsanity.com/paper/1901.10738/) (2019)
- [Unsupervised Representation Learning for Time Series with Temporal Neighborhood Coding](https://mlsanity.com/paper/2106.00750/) (2021)
- [Time2Vec: Learning a Vector Representation of Time](https://mlsanity.com/paper/1907.05321/) (2019)
- [MINIROCKET: A Very Fast (Almost) Deterministic Transform for Time Series Classification](https://mlsanity.com/paper/2012.08791/) (2020)
- [CLOCS: Contrastive Learning of Cardiac Signals Across Space, Time, and Patients](https://mlsanity.com/paper/2005.13249/) (2020)
- [Time-Series Anomaly Detection Service at Microsoft](https://mlsanity.com/paper/1906.03821/) (2019)
- [Uncovering the structure of clinical EEG signals with self-supervised learning](https://mlsanity.com/paper/2007.16104/) (2020)
- [Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects](https://mlsanity.com/paper/2306.10125/) (2023)
- [Towards a Rigorous Evaluation of Time-series Anomaly Detection](https://mlsanity.com/paper/2109.05257/) (2021)
- [Self-Supervised Transformer for Sparse and Irregularly Sampled Multivariate Clinical Time-Series](https://mlsanity.com/paper/2107.14293/) (2021)
- [Subject-Aware Contrastive Learning for Biosignals](https://mlsanity.com/paper/2007.04871/) (2020)
- [SOM-VAE: Interpretable Discrete Representation Learning on Time Series](https://mlsanity.com/paper/1806.02199/) (2018)
- [Time Series Change Point Detection with Self-Supervised Contrastive Predictive Coding](https://mlsanity.com/paper/2011.14097/) (2020)
- [TimeNet: Pre-trained deep recurrent neural network for time series classification](https://mlsanity.com/paper/1706.08838/) (2017)
- [Autowarp: Learning a Warping Distance from Unlabeled Time Series Using Sequence Autoencoders](https://mlsanity.com/paper/1810.10107/) (2018)
- [Random Warping Series: A Random Features Method for Time-Series Embedding](https://mlsanity.com/paper/1809.05259/) (2018)

## Built on by (most cited first, 19 of 19)

- [UniTS: A Unified Multi-Task Time Series Model](https://mlsanity.com/paper/2403.00131/) (2024)
- [Universal Time-Series Representation Learning: A Survey](https://mlsanity.com/paper/2401.03717/) (2024)
- [Abstracted Shapes as Tokens -- A Generalizable and Interpretable Model for Time-series Classification](https://mlsanity.com/paper/2411.01006/) (2024)
- [TSPulse: Tiny Pre-Trained Models with Disentangled Representations for Rapid Time-Series Analysis](https://mlsanity.com/paper/2505.13033/) (2025)
- [Giving Sensors a Voice: Multimodal JEPA for Semantic Time-Series Embeddings](https://mlsanity.com/paper/2605.31580/) (2026)
- [Toward Physics-guided Time Series Embedding](https://mlsanity.com/paper/2410.06651/) (2024)
- [Pre-training Epidemic Time Series Forecasters with Compartmental Prototypes](https://mlsanity.com/paper/2502.03393/) (2025)
- [Nonlinear time-series embedding by monotone variational inequality](https://mlsanity.com/paper/2406.06894/) (2024)
- [Time to Embed: Unlocking Foundation Models for Time Series with Channel Descriptions](https://mlsanity.com/paper/2505.14543/) (2025)
- [Path Generation and Evaluation in Video Games: A Nonparametric Statistical Approach](https://mlsanity.com/paper/2506.03522/) (2025)
- [Leveraging External Factors in Household-Level Electrical Consumption Forecasting using Hypernetworks](https://mlsanity.com/paper/2506.14472/) (2025)
- [CF-JEPA: Mask-free forward prediction with asymmetric encoder utilization for time-series representation learning](https://mlsanity.com/paper/2606.07031/) (2026)
- [MFF-FTNet: Multi-scale Feature Fusion across Frequency and Temporal Domains for Time Series Forecasting](https://mlsanity.com/paper/2411.17382/) (2024)
- [PLanTS: Periodicity-aware Latent-state Representation Learning for Multivariate Time Series](https://mlsanity.com/paper/2509.05478/) (2025)
- [Fed-REACT: Federated Representation Learning for Heterogeneous and Evolving Data](https://mlsanity.com/paper/2509.07198/) (2025)
- [Optimizing the Training Diet: Data Mixture Search for Robust Time Series Forecasting](https://mlsanity.com/paper/2512.11546/) (2025)
- [A Two-Channel F-Transform Representation for Early Trajectory Characterization in Iterated Correlation Dynamics](https://mlsanity.com/paper/2606.05462/) (2026)
- [SL-S4Wave: Self-Supervised Learning of Physiological Waveforms with Structured State Space Models](https://mlsanity.com/paper/2606.19888/) (2026)
- [CORD: Learning Reusable Degradation Representations Across Heterogeneous Physical Systems](https://mlsanity.com/paper/2609.39784/) (2026)

## Abstract (arXiv)

Multivariate time series present challenges to standard machine learning techniques, as they are often unlabeled, high dimensional, noisy, and contain missing data. To address this, we propose T-Rep, a self-supervised method to learn time series representations at a timestep granularity. T-Rep learns vector embeddings of time alongside its feature extractor, to extract temporal features such as trend, periodicity, or distribution shifts from the signal. These time-embeddings are leveraged in pretext tasks, to incorporate smooth and fine-grained temporal dependencies in the representations, as well as reinforce robustness to missing data. We evaluate T-Rep on downstream classification, forecasting, and anomaly detection tasks. It is compared to existing self-supervised algorithms for time series, which it outperforms in all three tasks. We test T-Rep in missing data regimes, where it proves more resilient than its counterparts. Finally, we provide latent space visualisation experiments, highlighting the interpretability of the learned representations.