# Heterogeneous Contrastive Learning

arXiv: https://arxiv.org/abs/2105.09401 · HTML: https://mlsanity.com/paper/2105.09401/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 52 references that are in the map.
- First posted on arXiv 19 May 2021.

## Builds on (most cited first, 22 of 22)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Large Batch Training of Convolutional Networks](https://mlsanity.com/paper/1708.03888/) (2017)
- [Cross-stitch Networks for Multi-task Learning](https://mlsanity.com/paper/1604.03539/) (2016)
- [Debiased Contrastive Learning](https://mlsanity.com/paper/2007.00224/) (2020)
- [Learning Robust Representations via Multi-View Information Bottleneck](https://mlsanity.com/paper/2002.07017/) (2020)
- [On Deep Multi-View Representation Learning: Objectives and Optimization](https://mlsanity.com/paper/1602.01024/) (2016)
- [Fully-adaptive Feature Sharing in Multi-Task Networks with Applications in Person Attribute Classification](https://mlsanity.com/paper/1611.05377/) (2016)
- [HDMI: High-order Deep Multiplex Infomax](https://mlsanity.com/paper/2102.07810/) (2021)
- [Multi-Label Learning with Global and Local Label Correlation](https://mlsanity.com/paper/1704.01415/) (2017)
- [Multi-label Contrastive Predictive Coding](https://mlsanity.com/paper/2007.09852/) (2020)
- [Deep Co-Attention Network for Multi-View Subspace Learning](https://mlsanity.com/paper/2102.07751/) (2021)
- [Explain Images with Multimodal Recurrent Neural Networks](https://mlsanity.com/paper/1410.1090/) (2014)
- [Fast Multi-Instance Multi-Label Learning](https://mlsanity.com/paper/1310.2049/) (2013)
- [Multi-View Intact Space Learning](https://mlsanity.com/paper/1904.02340/) (2019)
- [Deep Multimodality Model for Multi-task Multi-view Learning](https://mlsanity.com/paper/1901.08723/) (2019)
- [Learning Sparse Feature Representations using Probabilistic Quadtrees and Deep Belief Nets](https://mlsanity.com/paper/1509.03413/) (2015)
- [Heterogeneous Contrastive Learning: Encoding Spatial Information for Compact Visual Representations](https://mlsanity.com/paper/2011.09941/) (2020)

## Built on by (most cited first, 15 of 15)

- [Language in the Flow of Time: Time-Series-Paired Texts Weaved into a Unified Temporal Narrative](https://mlsanity.com/paper/2502.08942/) (2025)
- [Heterogeneous Contrastive Learning for Foundation Models and Beyond](https://mlsanity.com/paper/2404.00225/) (2024)
- [MentorGNN: Deriving Curriculum for Pre-Training GNNs](https://mlsanity.com/paper/2208.09905/) (2022)
- [COIN: Co-Cluster Infomax for Bipartite Graphs](https://mlsanity.com/paper/2206.00006/) (2022)
- [PyG-SSL: A Graph Self-Supervised Learning Toolkit](https://mlsanity.com/paper/2412.21151/) (2024)
- [Can Graph Neural Networks Learn Language with Extremely Weak Text Supervision?](https://mlsanity.com/paper/2412.08174/) (2024)
- [Deeper-GXX: Deepening Arbitrary GNNs](https://mlsanity.com/paper/2110.13798/) (2021)
- [Fairness-aware Multi-view Clustering](https://mlsanity.com/paper/2302.05788/) (2023)
- [Automated Contrastive Learning Strategy Search for Time Series](https://mlsanity.com/paper/2403.12641/) (2024)
- [Instances and Labels: Hierarchy-aware Joint Supervised Contrastive Learning for Hierarchical Multi-Label Text Classification](https://mlsanity.com/paper/2310.05128/) (2023)
- [OWLEYE: Zero-Shot Learner for Cross-Domain Graph Data Anomaly Detection](https://mlsanity.com/paper/2601.19102/) (2026)
- [mCPT at SemEval-2023 Task 3: Multilingual Label-Aware Contrastive Pre-Training of Transformers for Few- and Zero-shot Framing Detection](https://mlsanity.com/paper/2303.09901/) (2023)
- [Online Multi-modal Root Cause Identification in Microservice Systems](https://mlsanity.com/paper/2410.10021/) (2024)
- [LAMP: Learnable Meta-Path Guided Adversarial Contrastive Learning for Heterogeneous Graphs](https://mlsanity.com/paper/2409.06323/) (2024)
- [Robust Biomedical Publication Type and Study Design Classification with Knowledge-Guided Perturbations](https://mlsanity.com/paper/2605.11502/) (2026)

## Abstract (arXiv)

With the advent of big data across multiple high-impact applications, we are often facing the challenge of complex heterogeneity. The newly collected data usually consist of multiple modalities and are characterized with multiple labels, thus exhibiting the co-existence of multiple types of heterogeneity. Although state-of-the-art techniques are good at modeling complex heterogeneity with sufficient label information, such label information can be quite expensive to obtain in real applications. Recently, researchers pay great attention to contrastive learning due to its prominent performance by utilizing rich unlabeled data. However, existing work on contrastive learning is not able to address the problem of false negative pairs, i.e., some `negative' pairs may have similar representations if they have the same label. To overcome the issues, in this paper, we propose a unified heterogeneous learning framework, which combines both the weighted unsupervised contrastive loss and the weighted supervised contrastive loss to model multiple types of heterogeneity. We first provide a theoretical analysis showing that the vanilla contrastive learning loss easily leads to the sub-optimal solution in the presence of false negative pairs, whereas the proposed weighted loss could automatically adjust the weight based on the similarity of the learned representations to mitigate this issue. Experimental results on real-world data sets demonstrate the effectiveness and the efficiency of the proposed framework modeling multiple types of heterogeneity.