# One-Shot Adaptation of GAN in Just One CLIP

arXiv: https://arxiv.org/abs/2203.09301 · HTML: https://mlsanity.com/paper/2203.09301/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 68 references that are in the map.
- First posted on arXiv 17 Mar 2022.

## Builds on (most cited first, 30 of 51)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](https://mlsanity.com/paper/2103.17249/) (2021)
- [Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation](https://mlsanity.com/paper/2008.00951/) (2020)
- [Interpreting the Latent Space of GANs for Semantic Face Editing](https://mlsanity.com/paper/1907.10786/) (2019)
- [Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?](https://mlsanity.com/paper/1904.03189/) (2019)
- [Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2007.15651/) (2020)
- [GANSpace: Discovering Interpretable GAN Controls](https://mlsanity.com/paper/2004.02546/) (2020)
- [Designing an Encoder for StyleGAN Image Manipulation](https://mlsanity.com/paper/2102.02766/) (2021)
- [PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models](https://mlsanity.com/paper/2003.03808/) (2020)
- [StyleRig: Rigging StyleGAN for 3D Control over Portrait Images](https://mlsanity.com/paper/2004.00121/) (2020)
- [DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effort](https://mlsanity.com/paper/2104.06490/) (2021)
- [Few-shot Image Generation via Cross-domain Correspondence](https://mlsanity.com/paper/2104.06820/) (2021)
- [CLIPstyler: Image Style Transfer with a Single Text Condition](https://mlsanity.com/paper/2112.00374/) (2021)
- [Arbitrary Style Transfer with Style-Attentional Networks](https://mlsanity.com/paper/1812.02342/) (2018)
- [GAN Prior Embedded Network for Blind Face Restoration in the Wild](https://mlsanity.com/paper/2105.06070/) (2021)
- [AdaAttN: Revisit Attention Mechanism in Arbitrary Neural Style Transfer](https://mlsanity.com/paper/2108.03647/) (2021)
- [Adversarial Latent Autoencoders](https://mlsanity.com/paper/2004.04467/) (2020)
- [Transferring GANs: generating images from limited data](https://mlsanity.com/paper/1805.01677/) (2018)
- [Freeze the Discriminator: a Simple Baseline for Fine-Tuning GANs](https://mlsanity.com/paper/2002.10964/) (2020)
- [Few-shot Image Generation with Elastic Weight Consolidation](https://mlsanity.com/paper/2012.02780/) (2020)

## Built on by (most cited first, 30 of 31)

- [ZipLoRA: Any Subject in Any Style by Effectively Merging LoRAs](https://mlsanity.com/paper/2311.13600/) (2023)
- [Diffusion-based Image Translation using Disentangled Style and Content Representation](https://mlsanity.com/paper/2209.15264/) (2022)
- [Contrastive Denoising Score for Text-guided Latent Diffusion Image Editing](https://mlsanity.com/paper/2311.18608/) (2023)
- [Unified Language-driven Zero-shot Domain Adaptation](https://mlsanity.com/paper/2404.07155/) (2024)
- [UnZipLoRA: Separating Content and Style from a Single Image](https://mlsanity.com/paper/2412.04465/) (2024)
- [Towards Diverse and Faithful One-shot Adaption of Generative Adversarial Networks](https://mlsanity.com/paper/2207.08736/) (2022)
- [Generalized One-shot Domain Adaptation of Generative Adversarial Networks](https://mlsanity.com/paper/2209.03665/) (2022)
- [A Survey on Generative Modeling with Limited Data, Few Shots, and Zero Shot](https://mlsanity.com/paper/2307.14397/) (2023)
- [DomainStudio: Fine-Tuning Diffusion Models for Domain-Driven Image Generation using Limited Data](https://mlsanity.com/paper/2306.14153/) (2023)
- [Scenimefy: Learning to Craft Anime Scene via Semi-Supervised Image-to-Image Translation](https://mlsanity.com/paper/2308.12968/) (2023)
- [One-Shot Structure-Aware Stylized Image Synthesis](https://mlsanity.com/paper/2402.17275/) (2024)
- [Deformable One-shot Face Stylization via DINO Semantic Guidance](https://mlsanity.com/paper/2403.00459/) (2024)
- [Few-shot Image Generation via Masked Discrimination](https://mlsanity.com/paper/2210.15194/) (2022)
- [WeditGAN: Few-Shot Image Generation via Latent Space Relocation](https://mlsanity.com/paper/2305.06671/) (2023)
- [Contrastive Test-Time Composition of Multiple LoRA Models for Image Generation](https://mlsanity.com/paper/2403.19776/) (2024)
- [Image Synthesis under Limited Data: A Survey and Taxonomy](https://mlsanity.com/paper/2307.16879/) (2023)
- [Peer is Your Pillar: A Data-unbalanced Conditional GANs for Few-shot Image Generation](https://mlsanity.com/paper/2311.08217/) (2023)
- [MultiStyleGAN: Multiple One-shot Image Stylizations using a Single GAN](https://mlsanity.com/paper/2210.04120/) (2022)
- [Improving Diffusion-based Image Translation using Asymmetric Gradient Guidance](https://mlsanity.com/paper/2306.04396/) (2023)
- [One-shot Generative Domain Adaptation in 3D GANs](https://mlsanity.com/paper/2410.08824/) (2024)
- [Contrastive Learning Guided Latent Diffusion Model for Image-to-Image Translation](https://mlsanity.com/paper/2503.20484/) (2025)
- [Nickel and Diming Your GAN: A Dual-Method Approach to Enhancing GAN Efficiency via Knowledge Distillation](https://mlsanity.com/paper/2405.11614/) (2024)
- [Few-shot 3D Shape Generation](https://mlsanity.com/paper/2305.11664/) (2023)
- [Multi-Modal Face Stylization with a Generative Prior](https://mlsanity.com/paper/2305.18009/) (2023)
- [Few-Shot Domain Adaptation for Learned Image Compression](https://mlsanity.com/paper/2409.11111/) (2024)
- [HyperGAN-CLIP: A Unified Framework for Domain Adaptation, Image Synthesis and Manipulation](https://mlsanity.com/paper/2411.12832/) (2024)
- [One-Shot Stylization for Full-Body Human Images](https://mlsanity.com/paper/2304.06917/) (2023)
- [Domain Adaptation based on Human Feedback for Enhancing Generative Model Denoising Abilities](https://mlsanity.com/paper/2308.00307/) (2023)
- [Customize StyleGAN with One Hand Sketch](https://mlsanity.com/paper/2310.18949/) (2023)
- [AIR: Zero-shot Generative Model Adaptation with Iterative Refinement](https://mlsanity.com/paper/2506.10895/) (2025)

## Abstract (arXiv)

There are many recent research efforts to fine-tune a pre-trained generator with a few target images to generate images of a novel domain. Unfortunately, these methods often suffer from overfitting or under-fitting when fine-tuned with a single target image. To address this, here we present a novel single-shot GAN adaptation method through unified CLIP space manipulations. Specifically, our model employs a two-step training strategy: reference image search in the source generator using a CLIP-guided latent optimization, followed by generator fine-tuning with a novel loss function that imposes CLIP space consistency between the source and adapted generators. To further improve the adapted model to produce spatially consistent samples with respect to the source generator, we also propose contrastive regularization for patchwise relationships in the CLIP space. Experimental results show that our model generates diverse outputs with the target texture and outperforms the baseline models both qualitatively and quantitatively. Furthermore, we show that our CLIP space manipulation strategy allows more effective attribute editing.