# LOHO: Latent Optimization of Hairstyles via Orthogonalization

arXiv: https://arxiv.org/abs/2103.03891 · HTML: https://mlsanity.com/paper/2103.03891/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 43 references that are in the map.
- First posted on arXiv 5 Mar 2021.

## Builds on (most cited first, 30 of 32)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation](https://mlsanity.com/paper/2008.00951/) (2020)
- [Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks](https://mlsanity.com/paper/1703.10593/) (2017)
- [Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?](https://mlsanity.com/paper/1904.03189/) (2019)
- [MaskGAN: Towards Diverse and Interactive Facial Image Manipulation](https://mlsanity.com/paper/1907.11922/) (2019)
- [Free-Form Image Inpainting with Gated Convolution](https://mlsanity.com/paper/1806.03589/) (2018)
- [Segmentation Transformer: Object-Contextual Representations for Semantic Segmentation](https://mlsanity.com/paper/1909.11065/) (2019)
- [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017)
- [In-Domain GAN Inversion for Real Image Editing](https://mlsanity.com/paper/2004.00049/) (2020)
- [Video-to-Video Synthesis](https://mlsanity.com/paper/1808.06601/) (2018)
- [Image2StyleGAN++: How to Edit the Embedded Images?](https://mlsanity.com/paper/1911.11544/) (2019)
- [Everybody Dance Now](https://mlsanity.com/paper/1808.07371/) (2018)
- [Few-Shot Adversarial Learning of Realistic Neural Talking Head Models](https://mlsanity.com/paper/1905.08233/) (2019)
- [Few-shot Video-to-Video Synthesis](https://mlsanity.com/paper/1910.12713/) (2019)
- [Hierarchical Multi-Scale Attention for Semantic Segmentation](https://mlsanity.com/paper/2005.10821/) (2020)
- [Adversarial Latent Autoencoders](https://mlsanity.com/paper/2004.04467/) (2020)
- [SC-FEGAN: Face Editing Generative Adversarial Network with User's Sketch and Color](https://mlsanity.com/paper/1902.06838/) (2019)
- [Neural Head Reenactment with Latent Pose Descriptors](https://mlsanity.com/paper/2004.12000/) (2020)
- [Graphonomy: Universal Human Parsing via Graph Transfer Learning](https://mlsanity.com/paper/1904.04536/) (2019)

## Built on by (most cited first, 30 of 30)

- [HairCLIP: Design Your Hair by Text and Reference Image](https://mlsanity.com/paper/2112.05142/) (2021)
- [Fine-Grained Face Swapping via Regional GAN Inversion](https://mlsanity.com/paper/2211.14068/) (2022)
- [RenderMe-360: A Large Digital Asset Library and Benchmarks Towards High-fidelity Head Avatars](https://mlsanity.com/paper/2305.13353/) (2023)
- [Stable-Hair: Real-World Hair Transfer via Diffusion Model](https://mlsanity.com/paper/2407.14078/) (2024)
- [StyleFusion: A Generative Model for Disentangling Spatial Segments](https://mlsanity.com/paper/2107.07437/) (2021)
- [Temporally Consistent Semantic Video Editing](https://mlsanity.com/paper/2206.10590/) (2022)
- [HairCLIPv2: Unifying Hair Editing via Proxy Feature Blending](https://mlsanity.com/paper/2310.10651/) (2023)
- [Style Your Hair: Latent Optimization for Pose-Invariant Hairstyle Transfer via Local-Style-Aware Hair Alignment](https://mlsanity.com/paper/2208.07765/) (2022)
- [Human-Object Interaction Detection Collaborated with Large Relation-driven Diffusion Models](https://mlsanity.com/paper/2410.20155/) (2024)
- [Barbershop: GAN-based Image Compositing using Segmentation Masks](https://mlsanity.com/paper/2106.01505/) (2021)
- [Diffusion-driven GAN Inversion for Multi-Modal Face Image Generation](https://mlsanity.com/paper/2405.04356/) (2024)
- [Exploring Gradient-based Multi-directional Controls in GANs](https://mlsanity.com/paper/2209.00698/) (2022)
- [HairFastGAN: Realistic and Robust Hair Transfer with a Fast Encoder-Based Approach](https://mlsanity.com/paper/2404.01094/) (2024)
- [HairFIT: Pose-Invariant Hairstyle Transfer via Flow-based Hair Alignment and Semantic-Region-Aware Inpainting](https://mlsanity.com/paper/2206.08585/) (2022)
- [IntereStyle: Encoding an Interest Region for Robust StyleGAN Inversion](https://mlsanity.com/paper/2209.10811/) (2022)
- [ManiCLIP: Multi-Attribute Face Manipulation from Text](https://mlsanity.com/paper/2210.00445/) (2022)
- [E4S: Fine-grained Face Swapping via Editing With Regional GAN Inversion](https://mlsanity.com/paper/2310.15081/) (2023)
- [StyleGAN Salon: Multi-View Latent Optimization for Pose-Invariant Hairstyle Transfer](https://mlsanity.com/paper/2304.02744/) (2023)
- [SegTalker: Segmentation-based Talking Face Generation with Mask-guided Local Editing](https://mlsanity.com/paper/2409.03605/) (2024)
- [HairDiffusion: Vivid Multi-Colored Hair Editing via Latent Diffusion](https://mlsanity.com/paper/2410.21789/) (2024)
- [Real-time Virtual-Try-On from a Single Example Image through Deep Inverse Graphics and Learned Differentiable Renderers](https://mlsanity.com/paper/2205.06305/) (2022)
- [gCoRF: Generative Compositional Radiance Fields](https://mlsanity.com/paper/2210.17344/) (2022)
- [Stable-Hair v2: Real-World Hair Transfer via Multiple-View Diffusion Model](https://mlsanity.com/paper/2507.07591/) (2025)
- [HyperEditor: Achieving Both Authenticity and Cross-Domain Capability in Image Editing via Hypernetworks](https://mlsanity.com/paper/2312.13537/) (2023)
- [Face sketch to photo translation using generative adversarial networks](https://mlsanity.com/paper/2110.12290/) (2021)
- [Efficient Hair Style Transfer with Generative Adversarial Networks](https://mlsanity.com/paper/2210.12524/) (2022)
- [Trained Latent Space Navigation to Prevent Lack of Photorealism in Generated Images on Style-based Models](https://mlsanity.com/paper/2310.00936/) (2023)
- [HairShifter: Consistent and High-Fidelity Video Hair Transfer via Anchor-Guided Animation](https://mlsanity.com/paper/2507.12758/) (2025)
- [HairPort: In-context 3D-aware Hair Import and Transfer for Images](https://mlsanity.com/paper/2606.12562/) (2026)
- [H-Adapter: Pose-Robust Hairstyle Transfer via Attention-Derived, Source-Aligned Hair Masks](https://mlsanity.com/paper/2606.25578/) (2026)

## Abstract (arXiv)

Hairstyle transfer is challenging due to hair structure differences in the source and target hair. Therefore, we propose Latent Optimization of Hairstyles via Orthogonalization (LOHO), an optimization-based approach using GAN inversion to infill missing hair structure details in latent space during hairstyle transfer. Our approach decomposes hair into three attributes: perceptual structure, appearance, and style, and includes tailored losses to model each of these attributes independently. Furthermore, we propose two-stage optimization and gradient orthogonalization to enable disentangled latent space optimization of our hair attributes. Using LOHO for latent space manipulation, users can synthesize novel photorealistic images by manipulating hair attributes either individually or jointly, transferring the desired attributes from reference hairstyles. LOHO achieves a superior FID compared with the current state-of-the-art (SOTA) for hairstyle transfer. Additionally, LOHO preserves the subject's identity comparably well according to PSNR and SSIM when compared to SOTA image embedding pipelines. Code is available at https://github.com/dukebw/LOHO.