# FakeCLR: Exploring Contrastive Learning for Solving Latent Discontinuity in Data-Efficient GANs

arXiv: https://arxiv.org/abs/2207.08630 · HTML: https://mlsanity.com/paper/2207.08630/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 51 references that are in the map.
- First posted on arXiv 18 Jul 2022.

## Builds on (most cited first, 30 of 41)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020)
- [Contrastive Learning with Hard Negative Samples](https://mlsanity.com/paper/2010.04592/) (2020)
- [Towards Principled Methods for Training Generative Adversarial Networks](https://mlsanity.com/paper/1701.04862/) (2017)
- [Hard Negative Mixing for Contrastive Learning](https://mlsanity.com/paper/2010.01028/) (2020)
- [Differentiable Augmentation for Data-Efficient GAN Training](https://mlsanity.com/paper/2006.10738/) (2020)
- [What Should Not Be Contrastive in Contrastive Learning](https://mlsanity.com/paper/2008.05659/) (2020)
- [Projected GANs Converge Faster](https://mlsanity.com/paper/2111.01007/) (2021)
- [Few-shot Image Generation via Cross-domain Correspondence](https://mlsanity.com/paper/2104.06820/) (2021)
- [RankSRGAN: Generative Adversarial Networks with Ranker for Image Super-Resolution](https://mlsanity.com/paper/1908.06382/) (2019)
- [Towards Faster and Stabilized GAN Training for High-fidelity Few-shot Image Synthesis](https://mlsanity.com/paper/2101.04775/) (2021)
- [Transferring GANs: generating images from limited data](https://mlsanity.com/paper/1805.01677/) (2018)
- [Freeze the Discriminator: a Simple Baseline for Fine-Tuning GANs](https://mlsanity.com/paper/2002.10964/) (2020)
- [Regularizing Generative Adversarial Networks under Limited Data](https://mlsanity.com/paper/2104.03310/) (2021)
- [MineGAN: effective knowledge transfer from GANs to target domains with few images](https://mlsanity.com/paper/1912.05270/) (2019)
- [Image Generation From Small Datasets via Batch Statistics Adaptation](https://mlsanity.com/paper/1904.01774/) (2019)
- [Ensembling Off-the-shelf Models for GAN Training](https://mlsanity.com/paper/2112.09130/) (2021)
- [Image Augmentations for GAN Training](https://mlsanity.com/paper/2006.02595/) (2020)
- [Pose Guided Human Video Generation](https://mlsanity.com/paper/1807.11152/) (2018)
- [TSIT: A Simple and Versatile Framework for Image-to-Image Translation](https://mlsanity.com/paper/2007.12072/) (2020)
- [Data-Efficient Instance Generation from Instance Discrimination](https://mlsanity.com/paper/2106.04566/) (2021)
- [Deceive D: Adaptive Pseudo Augmentation for GAN Training with Limited Data](https://mlsanity.com/paper/2111.06849/) (2021)
- [Training GANs with Stronger Augmentations via Contrastive Discriminator](https://mlsanity.com/paper/2103.09742/) (2021)

## Built on by (most cited first, 17 of 17)

- [Is Artificial Intelligence Generated Image Detection a Solved Problem?](https://mlsanity.com/paper/2505.12335/) (2025)
- [Infinite-ID: Identity-preserved Personalization via ID-semantics Decoupling Paradigm](https://mlsanity.com/paper/2403.11781/) (2024)
- [A Survey on Generative Modeling with Limited Data, Few Shots, and Zero Shot](https://mlsanity.com/paper/2307.14397/) (2023)
- [Domain Re-Modulation for Few-Shot Generative Domain Adaptation](https://mlsanity.com/paper/2302.02550/) (2023)
- [Image Synthesis under Limited Data: A Survey and Taxonomy](https://mlsanity.com/paper/2307.16879/) (2023)
- [Rejection Sampling IMLE: Designing Priors for Better Few-Shot Image Synthesis](https://mlsanity.com/paper/2409.17439/) (2024)
- [Pedestrian Trajectory Prediction Based on Social Interactions Learning With Random Weights](https://mlsanity.com/paper/2501.07711/) (2025)
- [Efficient Backdoor Attacks for Deep Neural Networks in Real-world Scenarios](https://mlsanity.com/paper/2306.08386/) (2023)
- [Peer is Your Pillar: A Data-unbalanced Conditional GANs for Few-shot Image Generation](https://mlsanity.com/paper/2311.08217/) (2023)
- [UGC: Unified GAN Compression for Efficient Image-to-Image Translation](https://mlsanity.com/paper/2309.09310/) (2023)
- [One-shot Generative Domain Adaptation in 3D GANs](https://mlsanity.com/paper/2410.08824/) (2024)
- [CHAIN: Enhancing Generalization in Data-Efficient GANs via lipsCHitz continuity constrAIned Normalization](https://mlsanity.com/paper/2404.00521/) (2024)
- [CAE-DFKD: Bridging the Transferability Gap in Data-Free Knowledge Distillation](https://mlsanity.com/paper/2504.21478/) (2025)
- [Improving Few-shot Image Generation by Structural Discrimination and Textural Modulation](https://mlsanity.com/paper/2308.16110/) (2023)
- [Latent Iterative Refinement Flow: A Geometric Constrained Approach for Few-Shot Generation](https://mlsanity.com/paper/2509.19903/) (2025)
- [How Noise Benefits AI-generated Image Detection](https://mlsanity.com/paper/2511.16136/) (2025)
- [HP-GAN: Harnessing pretrained networks for GAN improvement with FakeTwins and discriminator consistency](https://mlsanity.com/paper/2602.03039/) (2026)

## Abstract (arXiv)

Data-Efficient GANs (DE-GANs), which aim to learn generative models with a limited amount of training data, encounter several challenges for generating high-quality samples. Since data augmentation strategies have largely alleviated the training instability, how to further improve the generative performance of DE-GANs becomes a hotspot. Recently, contrastive learning has shown the great potential of increasing the synthesis quality of DE-GANs, yet related principles are not well explored. In this paper, we revisit and compare different contrastive learning strategies in DE-GANs, and identify (i) the current bottleneck of generative performance is the discontinuity of latent space; (ii) compared to other contrastive learning strategies, Instance-perturbation works towards latent space continuity, which brings the major improvement to DE-GANs. Based on these observations, we propose FakeCLR, which only applies contrastive learning on perturbed fake samples, and devises three related training techniques: Noise-related Latent Augmentation, Diversity-aware Queue, and Forgetting Factor of Queue. Our experimental results manifest the new state of the arts on both few-shot generation and limited-data generation. On multiple datasets, FakeCLR acquires more than 15% FID improvement compared to existing DE-GANs. Code is available at https://github.com/iceli1007/FakeCLR.