# Towards Domain-Agnostic Contrastive Learning

arXiv: https://arxiv.org/abs/2011.04419 · HTML: https://mlsanity.com/paper/2011.04419/

## Key facts

- Cited by 75 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 9 Nov 2020.

## Built on by (most cited first, 30 of 75)

- [Graph Self-Supervised Learning: A Survey](https://mlsanity.com/paper/2103.00111/) (2021)
- [Self-supervised Learning on Graphs: Contrastive, Generative,or Predictive](https://mlsanity.com/paper/2105.07342/) (2021)
- [An Empirical Study of Graph Contrastive Learning](https://mlsanity.com/paper/2109.01116/) (2021)
- [BYOL for Audio: Self-Supervised Learning for General-Purpose Audio Representation](https://mlsanity.com/paper/2103.06695/) (2021)
- [Representational Continuity for Unsupervised Continual Learning](https://mlsanity.com/paper/2110.06976/) (2021)
- [Extending the WILDS Benchmark for Unsupervised Adaptation](https://mlsanity.com/paper/2112.05090/) (2021)
- [Knowledge Graph Contrastive Learning for Recommendation](https://mlsanity.com/paper/2205.00976/) (2022)
- [Equivariant Contrastive Learning](https://mlsanity.com/paper/2111.00899/) (2021)
- [Rethinking the Augmentation Module in Contrastive Learning: Learning Hierarchical Augmentation Invariance with Expanded Views](https://mlsanity.com/paper/2206.00227/) (2022)
- [Representation Learning for Tabular Data: A Comprehensive Survey](https://mlsanity.com/paper/2504.16109/) (2025)
- [Your Contrastive Learning Is Secretly Doing Stochastic Neighbor Embedding](https://mlsanity.com/paper/2205.14814/) (2022)
- [A Simple Data Mixing Prior for Improving Self-Supervised Learning](https://mlsanity.com/paper/2206.07692/) (2022)
- [Data Augmentation of Contrastive Learning is Estimating Positive-incentive Noise](https://mlsanity.com/paper/2408.09929/) (2024)
- [Augmentations in Graph Contrastive Learning: Current Methodological Flaws & Towards Better Practices](https://mlsanity.com/paper/2111.03220/) (2021)
- [Boosting Discriminative Visual Representation Learning with Scenario-Agnostic Mixup](https://mlsanity.com/paper/2111.15454/) (2021)
- [Towards Understanding the Data Dependency of Mixup-style Training](https://mlsanity.com/paper/2110.07647/) (2021)
- [Finding Order in Chaos: A Novel Data Augmentation Method for Time Series in Contrastive Learning](https://mlsanity.com/paper/2309.13439/) (2023)
- [A Survey on Mixup Augmentations and Beyond](https://mlsanity.com/paper/2409.05202/) (2024)
- [Harnessing Hard Mixed Samples with Decoupled Regularizer](https://mlsanity.com/paper/2203.10761/) (2022)
- [Improving Medical Multi-modal Contrastive Learning with Expert Annotations](https://mlsanity.com/paper/2403.10153/) (2024)
- [MetAug: Contrastive Learning via Meta Feature Augmentation](https://mlsanity.com/paper/2203.05119/) (2022)
- [Fine-Grained Classification with Noisy Labels](https://mlsanity.com/paper/2303.02404/) (2023)
- [Graph Mixup with Soft Alignments](https://mlsanity.com/paper/2306.06788/) (2023)
- [Learn Beneficial Noise as Graph Augmentation](https://mlsanity.com/paper/2505.19024/) (2025)
- [Provably Learning Diverse Features in Multi-View Data with Midpoint Mixup](https://mlsanity.com/paper/2210.13512/) (2022)
- [CL4CTR: A Contrastive Learning Framework for CTR Prediction](https://mlsanity.com/paper/2212.00522/) (2022)
- [Unsupervised Representation Learning for Time Series: A Review](https://mlsanity.com/paper/2308.01578/) (2023)
- [Self-supervised Representation Learning From Random Data Projectors](https://mlsanity.com/paper/2310.07756/) (2023)
- [Parametric Augmentation for Time Series Contrastive Learning](https://mlsanity.com/paper/2402.10434/) (2024)
- [Revisiting Contrastive Learning through the Lens of Neighborhood Component Analysis: an Integrated Framework](https://mlsanity.com/paper/2112.04468/) (2021)

## Abstract (arXiv)

Despite recent success, most contrastive self-supervised learning methods are domain-specific, relying heavily on data augmentation techniques that require knowledge about a particular domain, such as image cropping and rotation. To overcome such limitation, we propose a novel domain-agnostic approach to contrastive learning, named DACL, that is applicable to domains where invariances, and thus, data augmentation techniques, are not readily available. Key to our approach is the use of Mixup noise to create similar and dissimilar examples by mixing data samples differently either at the input or hidden-state levels. To demonstrate the effectiveness of DACL, we conduct experiments across various domains such as tabular data, images, and graphs. Our results show that DACL not only outperforms other domain-agnostic noising methods, such as Gaussian-noise, but also combines well with domain-specific methods, such as SimCLR, to improve self-supervised visual representation learning. Finally, we theoretically analyze our method and show advantages over the Gaussian-noise based contrastive learning approach.