# Hybrid Discriminative-Generative Training via Contrastive Learning

arXiv: https://arxiv.org/abs/2007.09070 · HTML: https://mlsanity.com/paper/2007.09070/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jul 2020.

## Built on by (most cited first, 24 of 24)

- [Your Diffusion Model is Secretly a Zero-Shot Classifier](https://mlsanity.com/paper/2303.16203/) (2023)
- [Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization](https://mlsanity.com/paper/2104.05833/) (2021)
- [Types of Out-of-Distribution Texts and How to Detect Them](https://mlsanity.com/paper/2109.06827/) (2021)
- [Consistent Representation Learning for Continual Relation Extraction](https://mlsanity.com/paper/2203.02721/) (2022)
- [Hybrid Predictive Coding: Inferring, Fast and Slow](https://mlsanity.com/paper/2204.02169/) (2022)
- [Breaking Down Out-of-Distribution Detection: Many Methods Based on OOD Training Data Estimate a Combination of the Same Core Quantities](https://mlsanity.com/paper/2206.09880/) (2022)
- [Joint Optimization of Ranking and Calibration with Contextualized Hybrid Model](https://mlsanity.com/paper/2208.06164/) (2022)
- [Do We Really Need to Learn Representations from In-domain Data for Outlier Detection?](https://mlsanity.com/paper/2105.09270/) (2021)
- [CADet: Fully Self-Supervised Out-Of-Distribution Detection With Contrastive Learning](https://mlsanity.com/paper/2210.01742/) (2022)
- [Learning Probabilistic Models from Generator Latent Spaces with Hat EBM](https://mlsanity.com/paper/2210.16486/) (2022)
- [ELM: Embedding and Logit Margins for Long-Tail Learning](https://mlsanity.com/paper/2204.13208/) (2022)
- [Multiscale Flow for Robust and Optimal Cosmological Analysis](https://mlsanity.com/paper/2306.04689/) (2023)
- [Hybrid Generative-Contrastive Representation Learning](https://mlsanity.com/paper/2106.06162/) (2021)
- [GEDI: GEnerative and DIscriminative Training for Self-Supervised Learning](https://mlsanity.com/paper/2212.13425/) (2022)
- [Conjugate Energy-Based Models](https://mlsanity.com/paper/2106.13798/) (2021)
- [Unifying Self-Supervised Clustering and Energy-Based Models](https://mlsanity.com/paper/2401.00873/) (2023)
- [AutoSciDACT: Automated Scientific Discovery through Contrastive Embedding and Hypothesis Testing](https://mlsanity.com/paper/2510.21935/) (2025)
- [A Unified View of cGANs with and without Classifiers](https://mlsanity.com/paper/2111.01035/) (2021)
- [How to Construct Energy for Images? Denoising Autoencoder Can Be Energy Based Model](https://mlsanity.com/paper/2303.03887/) (2023)
- [Classification-Denoising Networks](https://mlsanity.com/paper/2410.03505/) (2024)
- [Stabilized training of joint energy-based models and their practical applications](https://mlsanity.com/paper/2303.04187/) (2023)
- [DORIC : Domain Robust Fine-Tuning for Open Intent Clustering through Dependency Parsing](https://mlsanity.com/paper/2303.09827/) (2023)
- [Projected Belief Networks With Discriminative Alignment for Acoustic Event Classification: Rivaling State of the Art CNNs](https://mlsanity.com/paper/2401.11199/) (2024)
- [Contrastive Conformal Sets](https://mlsanity.com/paper/2603.26261/) (2026)

## Abstract (arXiv)

Contrastive learning and supervised learning have both seen significant progress and success. However, thus far they have largely been treated as two separate objectives, brought together only by having a shared neural network. In this paper we show that through the perspective of hybrid discriminative-generative training of energy-based models we can make a direct connection between contrastive learning and supervised learning. Beyond presenting this unified view, we show our specific choice of approximation of the energy-based loss outperforms the existing practice in terms of classification accuracy of WideResNet on CIFAR-10 and CIFAR-100. It also leads to improved performance on robustness, out-of-distribution detection, and calibration.