# Do Generated Data Always Help Contrastive Learning?

arXiv: https://arxiv.org/abs/2403.12448 · HTML: https://mlsanity.com/paper/2403.12448/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 46 of its 51 references that are in the map.
- First posted on arXiv 19 Mar 2024.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 46)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [An Empirical Study of Training Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.02057/) (2021)
- [Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere](https://mlsanity.com/paper/2005.10242/) (2020)
- [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [What Makes for Good Views for Contrastive Learning?](https://mlsanity.com/paper/2005.10243/) (2020)
- [Label-Efficient Semantic Segmentation with Diffusion Models](https://mlsanity.com/paper/2112.03126/) (2021)
- [Synthetic Data from Diffusion Models Improves ImageNet Classification](https://mlsanity.com/paper/2304.08466/) (2023)
- [Is synthetic data from generative models ready for image recognition?](https://mlsanity.com/paper/2210.07574/) (2022)
- [Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss](https://mlsanity.com/paper/2106.04156/) (2021)
- [Understanding self-supervised Learning Dynamics without Contrastive Pairs](https://mlsanity.com/paper/2102.06810/) (2021)
- [Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models](https://mlsanity.com/paper/2103.04922/) (2021)
- [StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual Representation Learners](https://mlsanity.com/paper/2306.00984/) (2023)
- [Solo-learn: A Library of Self-supervised Methods for Visual Representation Learning](https://mlsanity.com/paper/2108.01775/) (2021)
- [On the duality between contrastive and non-contrastive self-supervised learning](https://mlsanity.com/paper/2206.02574/) (2022)
- [Chaos is a Ladder: A New Theoretical Understanding of Contrastive Learning via Augmentation Overlap](https://mlsanity.com/paper/2203.13457/) (2022)
- [Generative Models as a Data Source for Multiview Representation Learning](https://mlsanity.com/paper/2106.05258/) (2021)
- [Understanding Contrastive Learning Requires Incorporating Inductive Biases](https://mlsanity.com/paper/2202.14037/) (2022)
- [Learning to See by Looking at Noise](https://mlsanity.com/paper/2106.05963/) (2021)

## Built on by (most cited first, 25 of 25)

- [Facilitating Multi-turn Function Calling for LLMs via Compositional Instruction Tuning](https://mlsanity.com/paper/2410.12952/) (2024)
- [FreeTumor: Advance Tumor Segmentation via Large-Scale Tumor Synthesis](https://mlsanity.com/paper/2406.01264/) (2024)
- [Synth-Empathy: Towards High-Quality Synthetic Empathy Data](https://mlsanity.com/paper/2407.21669/) (2024)
- [QPO: Query-dependent Prompt Optimization via Multi-Loop Offline Reinforcement Learning](https://mlsanity.com/paper/2408.10504/) (2024)
- [CC-Diff: Enhancing Contextual Coherence in Remote Sensing Image Synthesis](https://mlsanity.com/paper/2412.08464/) (2024)
- [SynthVLM: Towards High-Quality and Efficient Synthesis of Image-Caption Datasets for Vision-Language Models](https://mlsanity.com/paper/2407.20756/) (2024)
- [Federated Learning for Medical Image Classification: A Comprehensive Benchmark](https://mlsanity.com/paper/2504.05238/) (2025)
- [Are Images Indistinguishable to Humans Also Indistinguishable to Classifiers?](https://mlsanity.com/paper/2405.18029/) (2024)
- [An Augmentation Overlap Theory of Contrastive Learning](https://mlsanity.com/paper/2511.03114/) (2025)
- [Improving EEG Classification Through Randomly Reassembling Original and Generated Data with Transformer-based Diffusion Models](https://mlsanity.com/paper/2407.20253/) (2024)
- [Ctrl-GenAug: Controllable Generative Augmentation for Medical Sequence Classification](https://mlsanity.com/paper/2409.17091/) (2024)
- [Multi-View Incongruity Learning for Multimodal Sarcasm Detection](https://mlsanity.com/paper/2412.00756/) (2024)
- [Robust Polyp Detection and Diagnosis through Compositional Prompt-Guided Diffusion Models](https://mlsanity.com/paper/2502.17951/) (2025)
- [CausalDiffTab: Mixed-Type Causal-Aware Diffusion for Tabular Data Generation](https://mlsanity.com/paper/2506.14206/) (2025)
- [MixDiff: Mixing Natural and Synthetic Images for Robust Self-Supervised Representations](https://mlsanity.com/paper/2406.12368/) (2024)
- [Artificial Intelligence for Biomedical Video Generation](https://mlsanity.com/paper/2411.07619/) (2024)
- [The Role of Video Generation in Enhancing Data-Limited Action Understanding](https://mlsanity.com/paper/2505.19495/) (2025)
- [How does Labeling Error Impact Contrastive Learning? A Perspective from Data Dimensionality Reduction](https://mlsanity.com/paper/2507.11161/) (2025)
- [Generation Properties of Stochastic Interpolation under Finite Training Set](https://mlsanity.com/paper/2509.21925/) (2025)
- [Hybrid Data-Free Knowledge Distillation](https://mlsanity.com/paper/2412.13525/) (2024)
- [Provably Near-Optimal Federated Ensemble Distillation with Negligible Overhead](https://mlsanity.com/paper/2502.06349/) (2025)
- [Federated Balanced Learning](https://mlsanity.com/paper/2601.14042/) (2026)
- [SimReg: Achieving Higher Performance in the Pretraining via Embedding Similarity Regularization](https://mlsanity.com/paper/2605.08809/) (2026)
- [DIPBox: A Multi-scale Testing Framework for Tracking Dataset Regeneration](https://mlsanity.com/paper/2606.21240/) (2026)
- [Post-Generation Curation of Synthetic Images via Homogeneous-Heterogeneous Splitting](https://mlsanity.com/paper/2607.02637/) (2026)

## Abstract (arXiv)

Contrastive Learning (CL) has emerged as one of the most successful paradigms for unsupervised visual representation learning, yet it often depends on intensive manual data augmentations. With the rise of generative models, especially diffusion models, the ability to generate realistic images close to the real data distribution has been well recognized. These generated high-equality images have been successfully applied to enhance contrastive representation learning, a technique termed ``data inflation''. However, we find that the generated data (even from a good diffusion model like DDPM) may sometimes even harm contrastive learning. We investigate the causes behind this failure from the perspective of both data inflation and data augmentation. For the first time, we reveal the complementary roles that stronger data inflation should be accompanied by weaker augmentations, and vice versa. We also provide rigorous theoretical explanations for these phenomena via deriving its generalization bounds under data inflation. Drawing from these insights, we propose Adaptive Inflation (AdaInf), a purely data-centric strategy without introducing any extra computation cost. On benchmark datasets, AdaInf can bring significant improvements for various contrastive learning methods. Notably, without using external data, AdaInf obtains 94.70% linear accuracy on CIFAR-10 with SimCLR, setting a new record that surpasses many sophisticated methods. Code is available at https://github.com/PKU-ML/adainf.