# 3D Guided Fine-Grained Face Manipulation

arXiv: https://arxiv.org/abs/1902.08900 · HTML: https://mlsanity.com/paper/1902.08900/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Feb 2019.

## Built on by (most cited first, 30 of 45)

- [MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Model](https://mlsanity.com/paper/2311.16498/) (2023)
- [Motion Representations for Articulated Animation](https://mlsanity.com/paper/2104.11280/) (2021)
- [DCFace: Synthetic Face Generation with Dual Condition Diffusion Model](https://mlsanity.com/paper/2304.07060/) (2023)
- [Protecting Celebrities from DeepFake with Identity Consistency Transformer](https://mlsanity.com/paper/2203.01318/) (2022)
- [DiffPoseTalk: Speech-Driven Stylistic 3D Facial Animation and Head Pose Generation via Diffusion Models](https://mlsanity.com/paper/2310.00434/) (2023)
- [OTAvatar: One-shot Talking Face Avatar with Controllable Tri-plane Rendering](https://mlsanity.com/paper/2303.14662/) (2023)
- [Real-World Robot Applications of Foundation Models: A Review](https://mlsanity.com/paper/2402.05741/) (2024)
- [Audio-Driven Co-Speech Gesture Video Generation](https://mlsanity.com/paper/2212.02350/) (2022)
- [Unsupervised Human Pose Estimation through Transforming Shape Templates](https://mlsanity.com/paper/2105.04154/) (2021)
- [SOGAN: 3D-Aware Shadow and Occlusion Robust GAN for Makeup Transfer](https://mlsanity.com/paper/2104.10567/) (2021)
- [Controllable and Guided Face Synthesis for Unconstrained Face Recognition](https://mlsanity.com/paper/2207.10180/) (2022)
- [3DFaceShop: Explicitly Controllable 3D-Aware Portrait Generation](https://mlsanity.com/paper/2209.05434/) (2022)
- [Unsupervised Volumetric Animation](https://mlsanity.com/paper/2301.11326/) (2023)
- [Cross-Domain and Disentangled Face Manipulation with 3D Guidance](https://mlsanity.com/paper/2104.11228/) (2021)
- [CanonSwap: High-Fidelity and Consistent Video Face Swapping via Canonical Space Modulation](https://mlsanity.com/paper/2507.02691/) (2025)
- [50 Years of Automated Face Recognition](https://mlsanity.com/paper/2505.24247/) (2025)
- [Continuously Controllable Facial Expression Editing in Talking Face Videos](https://mlsanity.com/paper/2209.08289/) (2022)
- [Learning Dynamic Tetrahedra for High-Quality Talking Head Synthesis](https://mlsanity.com/paper/2402.17364/) (2024)
- [Data Synthesis with Diverse Styles for Face Recognition via 3DMM-Guided Diffusion](https://mlsanity.com/paper/2504.00430/) (2025)
- [Training and Tuning Generative Neural Radiance Fields for Attribute-Conditional 3D-Aware Face Generation](https://mlsanity.com/paper/2208.12550/) (2022)
- [Motion Transformer for Unsupervised Image Animation](https://mlsanity.com/paper/2209.14024/) (2022)
- [POCE: Pose-Controllable Expression Editing](https://mlsanity.com/paper/2304.08938/) (2023)
- [RPG-Palm: Realistic Pseudo-data Generation for Palmprint Recognition](https://mlsanity.com/paper/2307.14016/) (2023)
- [Learning Motion Refinement for Unsupervised Face Animation](https://mlsanity.com/paper/2310.13912/) (2023)
- [High Quality Human Image Animation using Regional Supervision and Motion Blur Condition](https://mlsanity.com/paper/2409.19580/) (2024)
- [Contrastive Decoupled Representation Learning and Regularization for Speech-Preserving Facial Expression Manipulation](https://mlsanity.com/paper/2504.05672/) (2025)
- [CrossDF: Improving Cross-Domain Deepfake Detection with Deep Information Decomposition](https://mlsanity.com/paper/2310.00359/) (2023)
- [Beyond 3DMM: Learning to Capture High-fidelity 3D Face Shape](https://mlsanity.com/paper/2204.04379/) (2022)
- [Multi-Domain Image-to-Image Translation with Adaptive Inference Graph](https://mlsanity.com/paper/2101.03806/) (2021)
- [Dynamic Neural Textures: Generating Talking-Face Videos with Continuously Controllable Expressions](https://mlsanity.com/paper/2204.06180/) (2022)

## Abstract (arXiv)

We present a method for fine-grained face manipulation. Given a face image with an arbitrary expression, our method can synthesize another arbitrary expression by the same person. This is achieved by first fitting a 3D face model and then disentangling the face into a texture and a shape. We then learn different networks in these two spaces. In the texture space, we use a conditional generative network to change the appearance, and carefully design input formats and loss functions to achieve the best results. In the shape space, we use a fully connected network to predict the accurate shapes and use the available depth data for supervision. Both networks are conditioned on expression coefficients rather than discrete labels, allowing us to generate an unlimited amount of expressions. We show the superiority of this disentangling approach through both quantitative and qualitative studies. In a user study, our method is preferred in 85% of cases when compared to the most recent work. When compared to the ground truth, annotators cannot reliably distinguish between our synthesized images and real images, preferring our method in 53% of the cases.