# SOGAN: 3D-Aware Shadow and Occlusion Robust GAN for Makeup Transfer arXiv: https://arxiv.org/abs/2104.10567 ยท HTML: https://mlsanity.com/paper/2104.10567/ ## Key facts - Cited by 22 papers in the map (data through 14 Oct 2026). - Builds on 20 of its 37 references that are in the map. - First posted on arXiv 21 Apr 2021. ## Builds on (most cited first, 20 of 20) - [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014) - [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017) - [Face2Face: Real-time Face Capture and Reenactment of RGB Videos](https://mlsanity.com/paper/2007.14808/) (2020) - [StyleRig: Rigging StyleGAN for 3D Control over Portrait Images](https://mlsanity.com/paper/2004.00121/) (2020) - [Deep Video Portraits](https://mlsanity.com/paper/1805.11714/) (2018) - [Disentangled and Controllable Face Image Generation via 3D Imitative-Contrastive Learning](https://mlsanity.com/paper/2004.11660/) (2020) - [Towards Fast, Accurate and Stable 3D Dense Face Alignment](https://mlsanity.com/paper/2009.09960/) (2020) - [Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network](https://mlsanity.com/paper/1803.07835/) (2018) - [Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wild](https://mlsanity.com/paper/1911.11130/) (2019) - [PSGAN: Pose and Expression Robust Spatial-Aware GAN for Customizable Makeup Transfer](https://mlsanity.com/paper/1909.06956/) (2019) - [LADN: Local Adversarial Disentangling Network for Facial Makeup and De-Makeup](https://mlsanity.com/paper/1904.11272/) (2019) - [Towards Large-Pose Face Frontalization in the Wild](https://mlsanity.com/paper/1704.06244/) (2017) - [3D Guided Fine-Grained Face Manipulation](https://mlsanity.com/paper/1902.08900/) (2019) - [Deep 3D Portrait from a Single Image](https://mlsanity.com/paper/2004.11598/) (2020) - [Face Super-Resolution Guided by 3D Facial Priors](https://mlsanity.com/paper/2007.09454/) (2020) - [DVG-Face: Dual Variational Generation for Heterogeneous Face Recognition](https://mlsanity.com/paper/2009.09399/) (2020) - [Semi-supervised Adversarial Learning to Generate Photorealistic Face Images of New Identities from 3D Morphable Model](https://mlsanity.com/paper/1804.03675/) (2018) - [Dual Variational Generation for Low-Shot Heterogeneous Face Recognition](https://mlsanity.com/paper/1903.10203/) (2019) - [Global and Local Consistent Wavelet-domain Age Synthesis](https://mlsanity.com/paper/1809.07764/) (2018) - [Reference-guided Face Component Editing](https://mlsanity.com/paper/2006.02051/) (2020) ## Built on by (most cited first, 22 of 22) - [DeltaEdit: Exploring Text-free Training for Text-Driven Image Manipulation](https://mlsanity.com/paper/2303.06285/) (2023) - [SSAT: A Symmetric Semantic-Aware Transformer Network for Makeup Transfer and Removal](https://mlsanity.com/paper/2112.03631/) (2021) - [BeautyREC: Robust, Efficient, and Content-preserving Makeup Transfer](https://mlsanity.com/paper/2212.05855/) (2022) - [Image-to-Image Translation with Disentangled Latent Vectors for Face Editing](https://mlsanity.com/paper/2301.04628/) (2023) - [Toward Tiny and High-quality Facial Makeup with Data Amplify Learning](https://mlsanity.com/paper/2403.15033/) (2024) - [SHMT: Self-supervised Hierarchical Makeup Transfer via Latent Diffusion Models](https://mlsanity.com/paper/2412.11058/) (2024) - [Reference-Based 3D-Aware Image Editing with Triplanes](https://mlsanity.com/paper/2404.03632/) (2024) - [Makeup Extraction of 3D Representation via Illumination-Aware Image Decomposition](https://mlsanity.com/paper/2302.13279/) (2023) - [Makeup Prior Models for 3D Facial Makeup Estimation and Applications](https://mlsanity.com/paper/2403.17761/) (2024) - [Language-Driven Interactive Shadow Detection](https://mlsanity.com/paper/2408.08543/) (2024) - [DRAN: Detailed Region-Adaptive Normalization for Conditional Image Synthesis](https://mlsanity.com/paper/2109.14525/) (2021) - [Semi-parametric Makeup Transfer via Semantic-aware Correspondence](https://mlsanity.com/paper/2203.02286/) (2022) - [3D-Aware Adversarial Makeup Generation for Facial Privacy Protection](https://mlsanity.com/paper/2306.14640/) (2023) - [BeautyBank: Encoding Facial Makeup in Latent Space](https://mlsanity.com/paper/2411.11231/) (2024) - [DeltaSpace: A Semantic-aligned Feature Space for Flexible Text-guided Image Editing](https://mlsanity.com/paper/2310.08785/) (2023) - [GT2-GS: Geometry-aware Texture Transfer for Gaussian Splatting](https://mlsanity.com/paper/2505.15208/) (2025) - [EvoMakeup: High-Fidelity and Controllable Makeup Editing with MakeupQuad](https://mlsanity.com/paper/2508.05994/) (2025) - [SARA: Controllable Makeup Transfer with Spatial Alignment and Region-Adaptive Normalization](https://mlsanity.com/paper/2311.16828/) (2023) - [FFHQ-Makeup: Paired Synthetic Makeup Dataset with Facial Consistency Across Multiple Styles](https://mlsanity.com/paper/2508.03241/) (2025) - [Gorgeous: Create Your Desired Character Facial Makeup from Any Ideas](https://mlsanity.com/paper/2404.13944/) (2024) - [DeshadowMamba: Deshadowing as 1D Sequential Similarity](https://mlsanity.com/paper/2510.24260/) (2025) - [MSN: Multi-directional Similarity Network for Hand-crafted and Deep-synthesized Copy-Move Forgery Detection](https://mlsanity.com/paper/2512.07110/) (2025) ## Abstract (arXiv) In recent years, virtual makeup applications have become more and more popular. However, it is still challenging to propose a robust makeup transfer method in the real-world environment. Current makeup transfer methods mostly work well on good-conditioned clean makeup images, but transferring makeup that exhibits shadow and occlusion is not satisfying. To alleviate it, we propose a novel makeup transfer method, called 3D-Aware Shadow and Occlusion Robust GAN (SOGAN). Given the source and the reference faces, we first fit a 3D face model and then disentangle the faces into shape and texture. In the texture branch, we map the texture to the UV space and design a UV texture generator to transfer the makeup. Since human faces are symmetrical in the UV space, we can conveniently remove the undesired shadow and occlusion from the reference image by carefully designing a Flip Attention Module (FAM). After obtaining cleaner makeup features from the reference image, a Makeup Transfer Module (MTM) is introduced to perform accurate makeup transfer. The qualitative and quantitative experiments demonstrate that our SOGAN not only achieves superior results in shadow and occlusion situations but also performs well in large pose and expression variations.