# What to Preserve and What to Transfer: Faithful, Identity-Preserving Diffusion-based Hairstyle Transfer

arXiv: https://arxiv.org/abs/2408.16450 · HTML: https://mlsanity.com/paper/2408.16450/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 36 references that are in the map.
- First posted on arXiv 29 Aug 2024.

## Builds on (most cited first, 26 of 26)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation](https://mlsanity.com/paper/2311.17117/) (2023)
- [BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation](https://mlsanity.com/paper/1808.00897/) (2018)
- [DensePose: Dense Human Pose Estimation In The Wild](https://mlsanity.com/paper/1802.00434/) (2018)
- [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017)
- [Paint by Example: Exemplar-based Image Editing with Diffusion Models](https://mlsanity.com/paper/2211.13227/) (2022)
- [AnyDoor: Zero-shot Object-level Image Customization](https://mlsanity.com/paper/2307.09481/) (2023)
- [VITON-HD: High-Resolution Virtual Try-On via Misalignment-Aware Normalization](https://mlsanity.com/paper/2103.16874/) (2021)
- [StableVITON: Learning Semantic Correspondence with Latent Diffusion Model for Virtual Try-On](https://mlsanity.com/paper/2312.01725/) (2023)
- [High-Resolution Virtual Try-On with Misalignment and Occlusion-Handled Conditions](https://mlsanity.com/paper/2206.14180/) (2022)
- [CelebV-Text: A Large-Scale Facial Text-Video Dataset](https://mlsanity.com/paper/2303.14717/) (2023)
- [HairCLIP: Design Your Hair by Text and Reference Image](https://mlsanity.com/paper/2112.05142/) (2021)
- [Stable-Hair: Real-World Hair Transfer via Diffusion Model](https://mlsanity.com/paper/2407.14078/) (2024)
- [K-Hairstyle: A Large-scale Korean Hairstyle Dataset for Virtual Hair Editing and Hairstyle Classification](https://mlsanity.com/paper/2102.06288/) (2021)
- [Reference-based Image Composition with Sketch via Structure-aware Diffusion Model](https://mlsanity.com/paper/2304.09748/) (2023)
- [Style Your Hair: Latent Optimization for Pose-Invariant Hairstyle Transfer via Local-Style-Aware Hair Alignment](https://mlsanity.com/paper/2208.07765/) (2022)
- [HairFastGAN: Realistic and Robust Hair Transfer with a Fast Encoder-Based Approach](https://mlsanity.com/paper/2404.01094/) (2024)
- [HairFIT: Pose-Invariant Hairstyle Transfer via Flow-based Hair Alignment and Semantic-Region-Aware Inpainting](https://mlsanity.com/paper/2206.08585/) (2022)
- [StyleGAN Salon: Multi-View Latent Optimization for Pose-Invariant Hairstyle Transfer](https://mlsanity.com/paper/2304.02744/) (2023)

## Built on by (most cited first, 8 of 8)

- [Stable-Hair v2: Real-World Hair Transfer via Multiple-View Diffusion Model](https://mlsanity.com/paper/2507.07591/) (2025)
- [ControlHair: Synergizing Physics Simulator and Video Diffusion for Controllable Dynamic Hair Rendering](https://mlsanity.com/paper/2509.21541/) (2025)
- [The Devil Is in the Leakage: A Disentangled Dual-Purification Framework for High-Fidelity Hairstyle Transfer](https://mlsanity.com/paper/2607.11281/) (2026)
- [Durian: Dual Reference Image-Guided Portrait Animation with Attribute Transfer](https://mlsanity.com/paper/2509.04434/) (2025)
- [UniBioTransfer: A Unified Framework for Multiple Biometrics Transfer](https://mlsanity.com/paper/2603.19637/) (2026)
- [AHS: Adaptive Head Synthesis via Synthetic Data Augmentations](https://mlsanity.com/paper/2604.15857/) (2026)
- [HairPort: In-context 3D-aware Hair Import and Transfer for Images](https://mlsanity.com/paper/2606.12562/) (2026)
- [H-Adapter: Pose-Robust Hairstyle Transfer via Attention-Derived, Source-Aligned Hair Masks](https://mlsanity.com/paper/2606.25578/) (2026)

## Abstract (arXiv)

Hairstyle transfer is a challenging task in the image editing field that modifies the hairstyle of a given face image while preserving its other appearance and background features. The existing hairstyle transfer approaches heavily rely on StyleGAN, which is pre-trained on cropped and aligned face images. Hence, they struggle to generalize under challenging conditions such as extreme variations of head poses or focal lengths. To address this issue, we propose a one-stage hairstyle transfer diffusion model, HairFusion, that applies to real-world scenarios. Specifically, we carefully design a hair-agnostic representation as the input of the model, where the original hair information is thoroughly eliminated. Next, we introduce a hair align cross-attention (Align-CA) to accurately align the reference hairstyle with the face image while considering the difference in their head poses. To enhance the preservation of the face image's original features, we leverage adaptive hair blending during the inference, where the output's hair regions are estimated by the cross-attention map in Align-CA and blended with non-hair areas of the face image. Our experimental results show that our method achieves state-of-the-art performance compared to the existing methods in preserving the integrity of both the transferred hairstyle and the surrounding features. The codes are available at https://github.com/cychungg/HairFusion