# Utilizing Expert Features for Contrastive Learning of Time-Series Representations

arXiv: https://arxiv.org/abs/2206.11517 · HTML: https://mlsanity.com/paper/2206.11517/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 41 references that are in the map.
- First posted on arXiv 23 Jun 2022.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere](https://mlsanity.com/paper/2005.10242/) (2020)
- [Relational Knowledge Distillation](https://mlsanity.com/paper/1904.05068/) (2019)
- [What Makes for Good Views for Contrastive Learning?](https://mlsanity.com/paper/2005.10243/) (2020)
- [Temporal Convolutional Networks for Action Segmentation and Detection](https://mlsanity.com/paper/1611.05267/) (2016)
- [Understanding the Behaviour of Contrastive Loss](https://mlsanity.com/paper/2012.09740/) (2020)
- [Debiased Contrastive Learning](https://mlsanity.com/paper/2007.00224/) (2020)
- [Time-Series Representation Learning via Temporal and Contextual Contrasting](https://mlsanity.com/paper/2106.14112/) (2021)
- [Keystroke Dynamics as Part of Lifelogging](https://mlsanity.com/paper/2010.10144/) (2020)
- [Unsupervised Scalable Representation Learning for Multivariate Time Series](https://mlsanity.com/paper/1901.10738/) (2019)
- [Unsupervised Representation Learning for Time Series with Temporal Neighborhood Coding](https://mlsanity.com/paper/2106.00750/) (2021)
- [Proxy Anchor Loss for Deep Metric Learning](https://mlsanity.com/paper/2003.13911/) (2020)
- [Harnessing Deep Neural Networks with Logic Rules](https://mlsanity.com/paper/1603.06318/) (2016)
- [An Empirical Survey of Data Augmentation for Time Series Classification with Neural Networks](https://mlsanity.com/paper/2007.15951/) (2020)
- [Weakly Supervised Contrastive Learning](https://mlsanity.com/paper/2110.04770/) (2021)
- [Viewmaker Networks: Learning Views for Unsupervised Representation Learning](https://mlsanity.com/paper/2010.07432/) (2020)
- [Learning Metrics from Teachers: Compact Networks for Image Embedding](https://mlsanity.com/paper/1904.03624/) (2019)
- [Deep Metric Learning Beyond Binary Supervision](https://mlsanity.com/paper/1904.09626/) (2019)
- [Task Programming: Learning Data Efficient Behavior Representations](https://mlsanity.com/paper/2011.13917/) (2020)
- [Embedding Transfer with Label Relaxation for Improved Metric Learning](https://mlsanity.com/paper/2103.14908/) (2021)
- [Contrastive Neural Processes for Self-Supervised Learning](https://mlsanity.com/paper/2110.13623/) (2021)
- [SleepPriorCL: Contrastive Representation Learning with Prior Knowledge-based Positive Mining and Adaptive Temperature for Sleep Staging](https://mlsanity.com/paper/2110.09966/) (2021)

## Built on by (most cited first, 17 of 17)

- [SimMTM: A Simple Pre-Training Framework for Masked Time-Series Modeling](https://mlsanity.com/paper/2302.00861/) (2023)
- [Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects](https://mlsanity.com/paper/2306.10125/) (2023)
- [Contrast Everything: A Hierarchical Contrastive Framework for Medical Time-Series](https://mlsanity.com/paper/2310.14017/) (2023)
- [A Survey on Time-Series Pre-Trained Models](https://mlsanity.com/paper/2305.10716/) (2023)
- [FOCAL: Contrastive Learning for Multimodal Time-Series Sensing Signals in Factorized Orthogonal Latent Space](https://mlsanity.com/paper/2310.20071/) (2023)
- [Label-efficient Time Series Representation Learning: A Review](https://mlsanity.com/paper/2302.06433/) (2023)
- [Unsupervised Representation Learning for Time Series: A Review](https://mlsanity.com/paper/2308.01578/) (2023)
- [Supervised Contrastive Learning with Hard Negative Samples](https://mlsanity.com/paper/2209.00078/) (2022)
- [Parametric Augmentation for Time Series Contrastive Learning](https://mlsanity.com/paper/2402.10434/) (2024)
- [Simple Contrastive Representation Learning for Time Series Forecasting](https://mlsanity.com/paper/2303.18205/) (2023)
- [Denoising-Aware Contrastive Learning for Noisy Time Series](https://mlsanity.com/paper/2406.04627/) (2024)
- [Self-Supervised Learning for Time Series: Contrastive or Generative?](https://mlsanity.com/paper/2403.09809/) (2024)
- [From Orthogonality to Dependency: Learning Disentangled Representation for Multi-Modal Time-Series Sensing Signals](https://mlsanity.com/paper/2405.16083/) (2024)
- [PreMixer: MLP-Based Pre-training Enhanced MLP-Mixers for Large-scale Traffic Forecasting](https://mlsanity.com/paper/2412.13607/) (2024)
- [Beyond Patient Invariance: Learning Cardiac Dynamics via Action-Conditioned JEPAs](https://mlsanity.com/paper/2604.22618/) (2026)
- [How does Labeling Error Impact Contrastive Learning? A Perspective from Data Dimensionality Reduction](https://mlsanity.com/paper/2507.11161/) (2025)
- [Learning Time-Series Representations by Hierarchical Uniformity-Tolerance Latent Balancing](https://mlsanity.com/paper/2510.01658/) (2025)

## Abstract (arXiv)

We present an approach that incorporates expert knowledge for time-series representation learning. Our method employs expert features to replace the commonly used data transformations in previous contrastive learning approaches. We do this since time-series data frequently stems from the industrial or medical field where expert features are often available from domain experts, while transformations are generally elusive for time-series data. We start by proposing two properties that useful time-series representations should fulfill and show that current representation learning approaches do not ensure these properties. We therefore devise ExpCLR, a novel contrastive learning approach built on an objective that utilizes expert features to encourage both properties for the learned representation. Finally, we demonstrate on three real-world time-series datasets that ExpCLR surpasses several state-of-the-art methods for both unsupervised and semi-supervised representation learning.