# A Framework For Contrastive Self-Supervised Learning And Designing A New Approach

arXiv: https://arxiv.org/abs/2009.00104 · HTML: https://mlsanity.com/paper/2009.00104/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Aug 2020.

## Built on by (most cited first, 30 of 52)

- [What to Hide from Your Students: Attention-Guided Masked Image Modeling](https://mlsanity.com/paper/2203.12719/) (2022)
- [SubTab: Subsetting Features of Tabular Data for Self-Supervised Representation Learning](https://mlsanity.com/paper/2110.04361/) (2021)
- [The Causal-Neural Connection: Expressiveness, Learnability, and Inference](https://mlsanity.com/paper/2107.00793/) (2021)
- [Self-supervised representation learning from 12-lead ECG data](https://mlsanity.com/paper/2103.12676/) (2021)
- [Neural Causal Models for Counterfactual Identification and Estimation](https://mlsanity.com/paper/2210.00035/) (2022)
- [A Comprehensive Study of Image Classification Model Sensitivity to Foregrounds, Backgrounds, and Visual Attributes](https://mlsanity.com/paper/2201.10766/) (2022)
- [Variational Wasserstein gradient flow](https://mlsanity.com/paper/2112.02424/) (2021)
- [Robust Contrastive Learning against Noisy Views](https://mlsanity.com/paper/2201.04309/) (2022)
- [Perfectly Balanced: Improving Transfer and Robustness of Supervised Contrastive Learning](https://mlsanity.com/paper/2204.07596/) (2022)
- [EM-driven unsupervised learning for efficient motion segmentation](https://mlsanity.com/paper/2201.02074/) (2022)
- [Server-Side Local Gradient Averaging and Learning Rate Acceleration for Scalable Split Learning](https://mlsanity.com/paper/2112.05929/) (2021)
- [Counterfactual Image Editing](https://mlsanity.com/paper/2403.09683/) (2024)
- [Latent Structure Mining with Contrastive Modality Fusion for Multimedia Recommendation](https://mlsanity.com/paper/2111.00678/) (2021)
- [A Survey on Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2302.09899/) (2023)
- [The Robustness Limits of SoTA Vision Models to Natural Variation](https://mlsanity.com/paper/2210.13604/) (2022)
- [Learn Beneficial Noise as Graph Augmentation](https://mlsanity.com/paper/2505.19024/) (2025)
- [Learning Representations with Contrastive Self-Supervised Learning for Histopathology Applications](https://mlsanity.com/paper/2112.05760/) (2021)
- [RELAX: Representation Learning Explainability](https://mlsanity.com/paper/2112.10161/) (2021)
- [Sample Efficient Detection and Classification of Adversarial Attacks via Self-Supervised Embeddings](https://mlsanity.com/paper/2108.13797/) (2021)
- [CAMBranch: Contrastive Learning with Augmented MILPs for Branching](https://mlsanity.com/paper/2402.03647/) (2024)
- [Tools and Practices for Responsible AI Engineering](https://mlsanity.com/paper/2201.05647/) (2022)
- [Deep Learning-based Framework for Automatic Cranial Defect Reconstruction and Implant Modeling](https://mlsanity.com/paper/2204.06310/) (2022)
- [TTAPS: Test-Time Adaption by Aligning Prototypes using Self-Supervision](https://mlsanity.com/paper/2205.08731/) (2022)
- [Using Self-Supervised Auxiliary Tasks to Improve Fine-Grained Facial Representation](https://mlsanity.com/paper/2105.06421/) (2021)
- [Sensor Data Augmentation by Resampling for Contrastive Learning in Human Activity Recognition](https://mlsanity.com/paper/2109.02054/) (2021)
- [Low-Shot Validation: Active Importance Sampling for Estimating Classifier Performance on Rare Categories](https://mlsanity.com/paper/2109.05720/) (2021)
- [Recur, Attend or Convolve? On Whether Temporal Modeling Matters for Cross-Domain Robustness in Action Recognition](https://mlsanity.com/paper/2112.12175/) (2021)
- [In-Distribution and Out-of-Distribution Self-supervised ECG Representation Learning for Arrhythmia Detection](https://mlsanity.com/paper/2304.06427/) (2023)
- [Cross-Lingual Open-Domain Question Answering with Answer Sentence Generation](https://mlsanity.com/paper/2110.07150/) (2021)
- [A Framework using Contrastive Learning for Classification with Noisy Labels](https://mlsanity.com/paper/2104.09563/) (2021)

## Abstract (arXiv)

Contrastive self-supervised learning (CSL) is an approach to learn useful representations by solving a pretext task that selects and compares anchor, negative and positive (APN) features from an unlabeled dataset. We present a conceptual framework that characterizes CSL approaches in five aspects (1) data augmentation pipeline, (2) encoder selection, (3) representation extraction, (4) similarity measure, and (5) loss function. We analyze three leading CSL approaches--AMDIM, CPC, and SimCLR--, and show that despite different motivations, they are special cases under this framework. We show the utility of our framework by designing Yet Another DIM (YADIM) which achieves competitive results on CIFAR-10, STL-10 and ImageNet, and is more robust to the choice of encoder and the representation extraction strategy. To support ongoing CSL research, we release the PyTorch implementation of this conceptual framework along with standardized implementations of AMDIM, CPC (V2), SimCLR, BYOL, Moco (V2) and YADIM.