# Soft Contrastive Learning for Time Series

arXiv: https://arxiv.org/abs/2312.16424 · HTML: https://mlsanity.com/paper/2312.16424/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 69 references that are in the map.
- First posted on arXiv 27 Dec 2023.

## Builds on (most cited first, 30 of 38)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [SimMIM: A Simple Framework for Masked Image Modeling](https://mlsanity.com/paper/2111.09886/) (2021)
- [DivideMix: Learning with Noisy Labels as Semi-supervised Learning](https://mlsanity.com/paper/2002.07394/) (2020)
- [Deep learning for time series classification: a review](https://mlsanity.com/paper/1809.04356/) (2018)
- [TS2Vec: Towards Universal Representation of Time Series](https://mlsanity.com/paper/2106.10466/) (2021)
- [A Transformer-based Framework for Multivariate Time Series Representation Learning](https://mlsanity.com/paper/2010.02803/) (2020)
- [The UCR Time Series Archive](https://mlsanity.com/paper/1810.07758/) (2018)
- [Time-Series Representation Learning via Temporal and Contextual Contrasting](https://mlsanity.com/paper/2106.14112/) (2021)
- [The UEA multivariate time series classification archive, 2018](https://mlsanity.com/paper/1811.00075/) (2018)
- [With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2104.14548/) (2021)
- [Unsupervised Scalable Representation Learning for Multivariate Time Series](https://mlsanity.com/paper/1901.10738/) (2019)
- [Self-Supervised Contrastive Pre-Training For Time Series via Time-Frequency Consistency](https://mlsanity.com/paper/2206.08496/) (2022)
- [Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applications](https://mlsanity.com/paper/1802.03903/) (2018)
- [Unsupervised Representation Learning for Time Series with Temporal Neighborhood Coding](https://mlsanity.com/paper/2106.00750/) (2021)
- [CLOCS: Contrastive Learning of Cardiac Signals Across Space, Time, and Patients](https://mlsanity.com/paper/2005.13249/) (2020)
- [Time-Series Anomaly Detection Service at Microsoft](https://mlsanity.com/paper/1906.03821/) (2019)
- [CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecasting](https://mlsanity.com/paper/2202.01575/) (2022)
- [Contrastive Masked Autoencoders are Stronger Vision Learners](https://mlsanity.com/paper/2207.13532/) (2022)
- [SimMTM: A Simple Pre-Training Framework for Masked Time-Series Modeling](https://mlsanity.com/paper/2302.00861/) (2023)
- [Subject-Aware Contrastive Learning for Biosignals](https://mlsanity.com/paper/2007.04871/) (2020)

## Built on by (most cited first, 30 of 51)

- [Universal Time-Series Representation Learning: A Survey](https://mlsanity.com/paper/2401.03717/) (2024)
- [RelCon: Relative Contrastive Learning for a Motion Foundation Model for Wearable Data](https://mlsanity.com/paper/2411.18822/) (2024)
- [MultiRC: Joint Learning for Time Series Anomaly Prediction and Detection with Multi-scale Reconstructive Contrast](https://mlsanity.com/paper/2410.15997/) (2024)
- [Segment, Shuffle, and Stitch: A Simple Layer for Improving Time-Series Representations](https://mlsanity.com/paper/2405.20082/) (2024)
- [AimTS: Augmented Series and Image Contrastive Learning for Time Series Classification](https://mlsanity.com/paper/2504.09993/) (2025)
- [Causality-Aware Contrastive Learning for Robust Multivariate Time-Series Anomaly Detection](https://mlsanity.com/paper/2506.03964/) (2025)
- [Contrastive Representation Learning for Predicting Solar Flares from Extremely Imbalanced Multivariate Time Series Data](https://mlsanity.com/paper/2410.00312/) (2024)
- [Generalization Analysis for Supervised Contrastive Representation Learning under Non-IID Settings](https://mlsanity.com/paper/2505.04937/) (2025)
- [Multi-View Contrastive Learning for Robust Domain Adaptation in Medical Time Series Analysis](https://mlsanity.com/paper/2506.22393/) (2025)
- [Deep Joint Distribution Optimal Transport for Universal Domain Adaptation on Time Series](https://mlsanity.com/paper/2503.11217/) (2025)
- [Channel-aware Contrastive Conditional Diffusion for Multivariate Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2410.02168/) (2024)
- [EXCON: Extreme Instance-based Contrastive Representation Learning of Severely Imbalanced Multivariate Time Series for Solar Flare Prediction](https://mlsanity.com/paper/2411.11249/) (2024)
- [Implicit Neural-Representation Learning for Elastic Deformable-Object Manipulations](https://mlsanity.com/paper/2505.00500/) (2025)
- [Decomposing and Revising What Language Models Generate](https://mlsanity.com/paper/2509.00765/) (2025)
- [FinZero: Launching Multi-modal Financial Time Series Forecast with Large Reasoning Model](https://mlsanity.com/paper/2509.08742/) (2025)
- [BenchHAR: Benchmarking Self-Supervised Learning for Generalizable Sensor-based Activity Recognition](https://mlsanity.com/paper/2605.08296/) (2026)
- [Structure-preserving contrastive learning for spatial time series](https://mlsanity.com/paper/2502.06380/) (2025)
- [LNUCB-TA: Linear-nonlinear Hybrid Bandit Learning with Temporal Attention](https://mlsanity.com/paper/2503.00387/) (2025)
- [Universal Domain Adaptation Benchmark for Time Series Data Representation](https://mlsanity.com/paper/2505.17899/) (2025)
- [Temporally-Aware Supervised Contrastive Learning for Polyp Counting in Colonoscopy](https://mlsanity.com/paper/2507.02493/) (2025)
- [How does Labeling Error Impact Contrastive Learning? A Perspective from Data Dimensionality Reduction](https://mlsanity.com/paper/2507.11161/) (2025)
- [Borrowing From the Future: Enhancing Early Risk Assessment through Contrastive Learning](https://mlsanity.com/paper/2508.11210/) (2025)
- [Learning Time-Series Representations by Hierarchical Uniformity-Tolerance Latent Balancing](https://mlsanity.com/paper/2510.01658/) (2025)
- [Cross-Sectional Asset Retrieval via Future-Aligned Soft Contrastive Learning](https://mlsanity.com/paper/2602.10711/) (2026)
- [Contrastive Learning Boosts Deterministic and Generative Models for Weather Data](https://mlsanity.com/paper/2603.24744/) (2026)
- [A Machine Learning Framework for Turbofan Health Estimation via Inverse Problem Formulation](https://mlsanity.com/paper/2604.08460/) (2026)
- [CF-JEPA: Mask-free forward prediction with asymmetric encoder utilization for time-series representation learning](https://mlsanity.com/paper/2606.07031/) (2026)
- [On the Influence of Shape, Texture and Color for Learning Semantic Segmentation](https://mlsanity.com/paper/2410.14878/) (2024)
- [Contrast All the Time: Learning Time Series Representation from Temporal Consistency](https://mlsanity.com/paper/2410.15416/) (2024)
- [Learning from Stochastic Teacher Representations Using Student-Guided Knowledge Distillation](https://mlsanity.com/paper/2504.14307/) (2025)

## Abstract (arXiv)

Contrastive learning has shown to be effective to learn representations from time series in a self-supervised way. However, contrasting similar time series instances or values from adjacent timestamps within a time series leads to ignore their inherent correlations, which results in deteriorating the quality of learned representations. To address this issue, we propose SoftCLT, a simple yet effective soft contrastive learning strategy for time series. This is achieved by introducing instance-wise and temporal contrastive loss with soft assignments ranging from zero to one. Specifically, we define soft assignments for 1) instance-wise contrastive loss by the distance between time series on the data space, and 2) temporal contrastive loss by the difference of timestamps. SoftCLT is a plug-and-play method for time series contrastive learning that improves the quality of learned representations without bells and whistles. In experiments, we demonstrate that SoftCLT consistently improves the performance in various downstream tasks including classification, semi-supervised learning, transfer learning, and anomaly detection, showing state-of-the-art performance. Code is available at this repository: https://github.com/seunghan96/softclt.