# HairStep: Transfer Synthetic to Real Using Strand and Depth Maps for Single-View 3D Hair Modeling

arXiv: https://arxiv.org/abs/2303.02700 · HTML: https://mlsanity.com/paper/2303.02700/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 49 references that are in the map.
- First posted on arXiv 5 Mar 2023.

## Builds on (most cited first, 22 of 22)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization](https://mlsanity.com/paper/2004.00452/) (2020)
- [Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture](https://mlsanity.com/paper/1411.4734/) (2014)
- [Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning](https://mlsanity.com/paper/1904.01786/) (2019)
- [Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2101.10979/) (2021)
- [Single-Image Depth Perception in the Wild](https://mlsanity.com/paper/1604.03901/) (2016)
- [RankSRGAN: Generative Adversarial Networks with Ranker for Image Super-Resolution](https://mlsanity.com/paper/1908.06382/) (2019)
- [Deep Convolutional Neural Fields for Depth Estimation from a Single Image](https://mlsanity.com/paper/1411.6387/) (2014)
- [Domain Adaptation with Auxiliary Target Domain-Oriented Classifier](https://mlsanity.com/paper/2007.04171/) (2020)
- [T2Net: Synthetic-to-Realistic Translation for Solving Single-Image Depth Estimation Tasks](https://mlsanity.com/paper/1808.01454/) (2018)
- [Geometry-Aware Symmetric Domain Adaptation for Monocular Depth Estimation](https://mlsanity.com/paper/1904.01870/) (2019)
- [DepthTransfer: Depth Extraction from Video Using Non-parametric Sampling](https://mlsanity.com/paper/2001.00987/) (2019)
- [Hallucinated-IQA: No-Reference Image Quality Assessment via Adversarial Learning](https://mlsanity.com/paper/1804.01681/) (2018)
- [NeuralHDHair: Automatic High-fidelity Hair Modeling from a Single Image Using Implicit Neural Representations](https://mlsanity.com/paper/2205.04175/) (2022)
- [DeepSketchHair: Deep Sketch-based 3D Hair Modeling](https://mlsanity.com/paper/1908.07198/) (2019)
- [A Neural Network for Detailed Human Depth Estimation from a Single Image](https://mlsanity.com/paper/1910.01275/) (2019)
- [MichiGAN: Multi-Input-Conditioned Hair Image Generation for Portrait Editing](https://mlsanity.com/paper/2010.16417/) (2020)
- [Deep Relative Attributes](https://mlsanity.com/paper/1512.04103/) (2015)

## Built on by (most cited first, 30 of 36)

- [GaussianHair: Hair Modeling and Rendering with Light-aware Gaussians](https://mlsanity.com/paper/2402.10483/) (2024)
- [Learning Disentangled Avatars with Hybrid 3D Representations](https://mlsanity.com/paper/2309.06441/) (2023)
- [Human Hair Reconstruction with Strand-Aligned 3D Gaussians](https://mlsanity.com/paper/2409.14778/) (2024)
- [HAAR: Text-Conditioned Generative Model of 3D Strand-based Human Hairstyles](https://mlsanity.com/paper/2312.11666/) (2023)
- [MonoHair: High-Fidelity Hair Modeling from a Monocular Video](https://mlsanity.com/paper/2403.18356/) (2024)
- [Perm: A Parametric Representation for Multi-Style 3D Hair Modeling](https://mlsanity.com/paper/2407.19451/) (2024)
- [DiffLocks: Generating 3D Hair from a Single Image using Diffusion Models](https://mlsanity.com/paper/2505.06166/) (2025)
- [Dr.Hair: Reconstructing Scalp-Connected Hair Strands without Pre-training via Differentiable Rendering of Line Segments](https://mlsanity.com/paper/2403.17496/) (2024)
- [Towards Unified 3D Hair Reconstruction from Single-View Portraits](https://mlsanity.com/paper/2409.16863/) (2024)
- [Im2Haircut: Single-view Strand-based Hair Reconstruction for Human Avatars](https://mlsanity.com/paper/2509.01469/) (2025)
- [3DGH: 3D Head Generation with Composable Hair and Face](https://mlsanity.com/paper/2506.20875/) (2025)
- [LUCAS: Layered Universal Codec Avatars](https://mlsanity.com/paper/2502.19739/) (2025)
- [Hairmony: Fairness-aware hairstyle classification](https://mlsanity.com/paper/2410.11528/) (2024)
- [StrandHead: Text to Hair-Disentangled 3D Head Avatars Using Human-Centric Priors](https://mlsanity.com/paper/2412.11586/) (2024)
- [CHARM: Control-point-based 3D Anime Hairstyle Auto-Regressive Modeling](https://mlsanity.com/paper/2509.21114/) (2025)
- [A Local Appearance Model for Volumetric Capture of Diverse Hairstyle](https://mlsanity.com/paper/2312.08679/) (2023)
- [HairDiffusion: Vivid Multi-Colored Hair Editing via Latent Diffusion](https://mlsanity.com/paper/2410.21789/) (2024)
- [StrandDesigner: Towards Practical Strand Generation with Sketch Guidance](https://mlsanity.com/paper/2508.01650/) (2025)
- [ControlHair: Synergizing Physics Simulator and Video Diffusion for Controllable Dynamic Hair Rendering](https://mlsanity.com/paper/2509.21541/) (2025)
- [PhysHead: Simulation-Ready Gaussian Head Avatars](https://mlsanity.com/paper/2604.06467/) (2026)
- [EMS: 3D Eyebrow Modeling from Single-view Images](https://mlsanity.com/paper/2309.12787/) (2023)
- [GeomHair: Reconstruction of Hair Strands from Colorless 3D Scans](https://mlsanity.com/paper/2505.05376/) (2025)
- [HairWeaver: Few-Shot Photorealistic Hair Motion Synthesis with Sim-to-Real Guided Video Diffusion](https://mlsanity.com/paper/2602.11117/) (2026)
- [HairOrbit: Multi-view Aware 3D Hair Modeling from Single Portraits](https://mlsanity.com/paper/2604.02867/) (2026)
- [CGHair: Compact Gaussian Hair Reconstruction with Card Clustering](https://mlsanity.com/paper/2604.03716/) (2026)
- [HairGPT: Strand-as-Language Autoregressive Modeling for Realistic 3D Hairstyle Synthesis](https://mlsanity.com/paper/2605.08824/) (2026)
- [HairLRM: Strand-based Hair Modeling via Large Reconstruction Models](https://mlsanity.com/paper/2606.15238/) (2026)
- [HairCS: Reconstructing Strand-Based Hair from Hair Cards](https://mlsanity.com/paper/2609.16465/) (2026)
- [The Devil is in the Details: Boosting Guided Depth Super-Resolution via Rethinking Cross-Modal Alignment and Aggregation](https://mlsanity.com/paper/2401.08123/) (2024)
- [Unsupervised 3D Braided Hair Reconstruction from a Single-View Image](https://mlsanity.com/paper/2506.23072/) (2025)

## Abstract (arXiv)

In this work, we tackle the challenging problem of learning-based single-view 3D hair modeling. Due to the great difficulty of collecting paired real image and 3D hair data, using synthetic data to provide prior knowledge for real domain becomes a leading solution. This unfortunately introduces the challenge of domain gap. Due to the inherent difficulty of realistic hair rendering, existing methods typically use orientation maps instead of hair images as input to bridge the gap. We firmly think an intermediate representation is essential, but we argue that orientation map using the dominant filtering-based methods is sensitive to uncertain noise and far from a competent representation. Thus, we first raise this issue up and propose a novel intermediate representation, termed as HairStep, which consists of a strand map and a depth map. It is found that HairStep not only provides sufficient information for accurate 3D hair modeling, but also is feasible to be inferred from real images. Specifically, we collect a dataset of 1,250 portrait images with two types of annotations. A learning framework is further designed to transfer real images to the strand map and depth map. It is noted that, an extra bonus of our new dataset is the first quantitative metric for 3D hair modeling. Our experiments show that HairStep narrows the domain gap between synthetic and real and achieves state-of-the-art performance on single-view 3D hair reconstruction.