# Towards Enhancing Time Series Contrastive Learning: A Dynamic Bad Pair Mining Approach

arXiv: https://arxiv.org/abs/2302.03357 · HTML: https://mlsanity.com/paper/2302.03357/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 34 references that are in the map.
- First posted on arXiv 7 Feb 2023.

## Builds on (most cited first, 26 of 26)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [What Makes for Good Views for Contrastive Learning?](https://mlsanity.com/paper/2005.10243/) (2020)
- [TS2Vec: Towards Universal Representation of Time Series](https://mlsanity.com/paper/2106.10466/) (2021)
- [Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics](https://mlsanity.com/paper/2009.10795/) (2020)
- [Time-Series Representation Learning via Temporal and Contextual Contrasting](https://mlsanity.com/paper/2106.14112/) (2021)
- [Self-Supervised Contrastive Pre-Training For Time Series via Time-Frequency Consistency](https://mlsanity.com/paper/2206.08496/) (2022)
- [Unsupervised Representation Learning for Time Series with Temporal Neighborhood Coding](https://mlsanity.com/paper/2106.00750/) (2021)
- [Deep Learning for ECG Analysis: Benchmarks and Insights from PTB-XL](https://mlsanity.com/paper/2004.13701/) (2020)
- [CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecasting](https://mlsanity.com/paper/2202.01575/) (2022)
- [Data Augmentation of Wearable Sensor Data for Parkinson's Disease Monitoring using Convolutional Neural Networks](https://mlsanity.com/paper/1706.00527/) (2017)
- [Toward Understanding the Feature Learning Process of Self-supervised Contrastive Learning](https://mlsanity.com/paper/2105.15134/) (2021)
- [Contrastive Learning with Stronger Augmentations](https://mlsanity.com/paper/2104.07713/) (2021)
- [Unsupervised Time-Series Representation Learning with Iterative Bilinear Temporal-Spectral Fusion](https://mlsanity.com/paper/2202.04770/) (2022)
- [Robust Audio-Visual Instance Discrimination](https://mlsanity.com/paper/2103.15916/) (2021)
- [Rethinking Minimal Sufficient Representation in Contrastive Learning](https://mlsanity.com/paper/2203.07004/) (2022)
- [How Does SimSiam Avoid Collapse Without Negative Samples? A Unified Understanding with Self-supervised Contrastive Learning](https://mlsanity.com/paper/2203.16262/) (2022)
- [Robust Contrastive Learning against Noisy Views](https://mlsanity.com/paper/2201.04309/) (2022)
- [Contrastive Learning for Unsupervised Domain Adaptation of Time Series](https://mlsanity.com/paper/2206.06243/) (2022)
- [Intra-Inter Subject Self-supervised Learning for Multivariate Cardiac Signals](https://mlsanity.com/paper/2109.08908/) (2021)
- [Neighborhood Contrastive Learning Applied to Online Patient Monitoring](https://mlsanity.com/paper/2106.05142/) (2021)

## Built on by (most cited first, 13 of 13)

- [ECG-FM: An Open Electrocardiogram Foundation Model](https://mlsanity.com/paper/2408.05178/) (2024)
- [Zero-Shot ECG Classification with Multimodal Learning and Test-time Clinical Knowledge Enhancement](https://mlsanity.com/paper/2403.06659/) (2024)
- [GEM: Empowering MLLM for Grounded ECG Understanding with Time Series and Images](https://mlsanity.com/paper/2503.06073/) (2025)
- [Learning the Unlearned: Mitigating Feature Suppression in Contrastive Learning](https://mlsanity.com/paper/2402.11816/) (2024)
- [Frequency-Masked Embedding Inference: A Non-Contrastive Approach for Time Series Representation Learning](https://mlsanity.com/paper/2412.20790/) (2024)
- [Robust Fairness Vision-Language Learning for Medical Image Analysis](https://mlsanity.com/paper/2505.03153/) (2025)
- [Self-Alignment Learning to Improve Myocardial Infarction Detection from Single-Lead ECG](https://mlsanity.com/paper/2509.19397/) (2025)
- [MEETI: A Multimodal ECG Dataset from MIMIC-IV-ECG with Signals, Images, Features and Interpretations](https://mlsanity.com/paper/2507.15255/) (2025)
- [RhythmBERT: A Self-Supervised Language Model Based on Latent Representations of ECG Waveforms for Heart Disease Detection](https://mlsanity.com/paper/2602.23060/) (2026)
- [ECG-Soup: Harnessing Multi-Layer Synergy for ECG Foundation Models](https://mlsanity.com/paper/2509.00102/) (2025)
- [PLanTS: Periodicity-aware Latent-state Representation Learning for Multivariate Time Series](https://mlsanity.com/paper/2509.05478/) (2025)
- [Open World Knowledge Aided Single-Cell Foundation Model with Robust Cross-Modal Cell-Language Pre-training](https://mlsanity.com/paper/2601.05648/) (2026)
- [Information-theoretic Multimodal Representation Learning for Electrocardiogram Signals](https://mlsanity.com/paper/2605.27583/) (2026)

## Abstract (arXiv)

Not all positive pairs are beneficial to time series contrastive learning. In this paper, we study two types of bad positive pairs that can impair the quality of time series representation learned through contrastive learning: the noisy positive pair and the faulty positive pair. We observe that, with the presence of noisy positive pairs, the model tends to simply learn the pattern of noise (Noisy Alignment). Meanwhile, when faulty positive pairs arise, the model wastes considerable amount of effort aligning non-representative patterns (Faulty Alignment). To address this problem, we propose a Dynamic Bad Pair Mining (DBPM) algorithm, which reliably identifies and suppresses bad positive pairs in time series contrastive learning. Specifically, DBPM utilizes a memory module to dynamically track the training behavior of each positive pair along training process. This allows us to identify potential bad positive pairs at each epoch based on their historical training behaviors. The identified bad pairs are subsequently down-weighted through a transformation module, thereby mitigating their negative impact on the representation learning process. DBPM is a simple algorithm designed as a lightweight plug-in without learnable parameters to enhance the performance of existing state-of-the-art methods. Through extensive experiments conducted on four large-scale, real-world time series datasets, we demonstrate DBPM's efficacy in mitigating the adverse effects of bad positive pairs.