# HairCLIP: Design Your Hair by Text and Reference Image

arXiv: https://arxiv.org/abs/2112.05142 · HTML: https://mlsanity.com/paper/2112.05142/

## Key facts

- Cited by 80 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 51 references that are in the map.
- First posted on arXiv 9 Dec 2021.

## Builds on (most cited first, 30 of 36)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks](https://mlsanity.com/paper/1908.02265/) (2019)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [LXMERT: Learning Cross-Modality Encoder Representations from Transformers](https://mlsanity.com/paper/1908.07490/) (2019)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](https://mlsanity.com/paper/2103.17249/) (2021)
- [VL-BERT: Pre-training of Generic Visual-Linguistic Representations](https://mlsanity.com/paper/1908.08530/) (2019)
- [CNN-generated images are surprisingly easy to spot... for now](https://mlsanity.com/paper/1912.11035/) (2019)
- [Florence: A New Foundation Model for Computer Vision](https://mlsanity.com/paper/2111.11432/) (2021)
- [Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation](https://mlsanity.com/paper/2008.00951/) (2020)
- [Interpreting the Latent Space of GANs for Semantic Face Editing](https://mlsanity.com/paper/1907.10786/) (2019)
- [Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?](https://mlsanity.com/paper/1904.03189/) (2019)
- [MaskGAN: Towards Diverse and Interactive Facial Image Manipulation](https://mlsanity.com/paper/1907.11922/) (2019)
- [Designing an Encoder for StyleGAN Image Manipulation](https://mlsanity.com/paper/2102.02766/) (2021)
- [CurricularFace: Adaptive Curriculum Learning Loss for Deep Face Recognition](https://mlsanity.com/paper/2004.00288/) (2020)
- [SEAN: Image Synthesis with Semantic Region-Adaptive Normalization](https://mlsanity.com/paper/1911.12861/) (2019)
- [On the "steerability" of generative adversarial networks](https://mlsanity.com/paper/1907.07171/) (2019)
- [GANalyze: Toward Visual Definitions of Cognitive Image Properties](https://mlsanity.com/paper/1906.10112/) (2019)
- [Editing in Style: Uncovering the Local Semantics of GANs](https://mlsanity.com/paper/2004.14367/) (2020)
- [A U-Net Based Discriminator for Generative Adversarial Networks](https://mlsanity.com/paper/2002.12655/) (2020)
- [SC-FEGAN: Face Editing Generative Adversarial Network with User's Sketch and Color](https://mlsanity.com/paper/1902.06838/) (2019)
- [Talk-to-Edit: Fine-Grained Facial Editing via Dialog](https://mlsanity.com/paper/2109.04425/) (2021)
- [Only a Matter of Style: Age Transformation Using a Style-Based Regression Model](https://mlsanity.com/paper/2102.02754/) (2021)
- [Efficient Semantic Image Synthesis via Class-Adaptive Normalization](https://mlsanity.com/paper/2012.04644/) (2020)
- [Language-Based Image Editing with Recurrent Attentive Models](https://mlsanity.com/paper/1711.06288/) (2017)

## Built on by (most cited first, 30 of 80)

- [Diffusion-based Image Translation using Disentangled Style and Content Representation](https://mlsanity.com/paper/2209.15264/) (2022)
- [NeRF-Art: Text-Driven Neural Radiance Fields Stylization](https://mlsanity.com/paper/2212.08070/) (2022)
- [TALL: Thumbnail Layout for Deepfake Video Detection](https://mlsanity.com/paper/2307.07494/) (2023)
- [Perceive-IR: Learning to Perceive Degradation Better for All-in-One Image Restoration](https://mlsanity.com/paper/2408.15994/) (2024)
- [ClipFace: Text-guided Editing of Textured 3D Morphable Models](https://mlsanity.com/paper/2212.01406/) (2022)
- [Multimodal Image Synthesis and Editing: The Generative AI Era](https://mlsanity.com/paper/2112.13592/) (2021)
- [RenderMe-360: A Large Digital Asset Library and Benchmarks Towards High-fidelity Head Avatars](https://mlsanity.com/paper/2305.13353/) (2023)
- [CLIP2Protect: Protecting Facial Privacy using Text-Guided Makeup via Adversarial Latent Search](https://mlsanity.com/paper/2306.10008/) (2023)
- [Multimodal Fake News Detection via CLIP-Guided Learning](https://mlsanity.com/paper/2205.14304/) (2022)
- [Text-Guided Generation and Editing of Compositional 3D Avatars](https://mlsanity.com/paper/2309.07125/) (2023)
- [High-fidelity Generalized Emotional Talking Face Generation with Multi-modal Emotion Space Learning](https://mlsanity.com/paper/2305.02572/) (2023)
- [Stable-Hair: Real-World Hair Transfer via Diffusion Model](https://mlsanity.com/paper/2407.14078/) (2024)
- [Learning Emotion Representations from Verbal and Nonverbal Communication](https://mlsanity.com/paper/2305.13500/) (2023)
- [DeltaEdit: Exploring Text-free Training for Text-Driven Image Manipulation](https://mlsanity.com/paper/2303.06285/) (2023)
- [Bootstrapping Multi-view Representations for Fake News Detection](https://mlsanity.com/paper/2206.05741/) (2022)
- [One Model to Edit Them All: Free-Form Text-Driven Image Manipulation with Semantic Modulations](https://mlsanity.com/paper/2210.07883/) (2022)
- [FreeFlux: Understanding and Exploiting Layer-Specific Roles in RoPE-Based MMDiT for Versatile Image Editing](https://mlsanity.com/paper/2503.16153/) (2025)
- [Personalized Image Generation with Deep Generative Models: A Decade Survey](https://mlsanity.com/paper/2502.13081/) (2025)
- [Tell2Design: A Dataset for Language-Guided Floor Plan Generation](https://mlsanity.com/paper/2311.15941/) (2023)
- [Delving StyleGAN Inversion for Image Editing: A Foundation Latent Space Viewpoint](https://mlsanity.com/paper/2211.11448/) (2022)
- [TextIR: A Simple Framework for Text-based Editable Image Restoration](https://mlsanity.com/paper/2302.14736/) (2023)
- [Pluralistic Aging Diffusion Autoencoder](https://mlsanity.com/paper/2303.11086/) (2023)
- [Language-guided Human Motion Synthesis with Atomic Actions](https://mlsanity.com/paper/2308.09611/) (2023)
- [HairCLIPv2: Unifying Hair Editing via Proxy Feature Blending](https://mlsanity.com/paper/2310.10651/) (2023)
- [ChatFace: Chat-Guided Real Face Editing via Diffusion Latent Space Manipulation](https://mlsanity.com/paper/2305.14742/) (2023)
- [Zero-Shot Text-to-Parameter Translation for Game Character Auto-Creation](https://mlsanity.com/paper/2303.01311/) (2023)
- [Multi-modal Fake News Detection on Social Media via Multi-grained Information Fusion](https://mlsanity.com/paper/2304.00827/) (2023)
- [PromptRR: Diffusion Models as Prompt Generators for Single Image Reflection Removal](https://mlsanity.com/paper/2402.02374/) (2024)
- [Bridging CLIP and StyleGAN through Latent Alignment for Image Editing](https://mlsanity.com/paper/2210.04506/) (2022)
- [CHATEDIT: Towards Multi-turn Interactive Facial Image Editing via Dialogue](https://mlsanity.com/paper/2303.11108/) (2023)

## Abstract (arXiv)

Hair editing is an interesting and challenging problem in computer vision and graphics. Many existing methods require well-drawn sketches or masks as conditional inputs for editing, however these interactions are neither straightforward nor efficient. In order to free users from the tedious interaction process, this paper proposes a new hair editing interaction mode, which enables manipulating hair attributes individually or jointly based on the texts or reference images provided by users. For this purpose, we encode the image and text conditions in a shared embedding space and propose a unified hair editing framework by leveraging the powerful image text representation capability of the Contrastive Language-Image Pre-Training (CLIP) model. With the carefully designed network structures and loss functions, our framework can perform high-quality hair editing in a disentangled manner. Extensive experiments demonstrate the superiority of our approach in terms of manipulation accuracy, visual realism of editing results, and irrelevant attribute preservation. Project repo is https://github.com/wty-ustc/HairCLIP.