# Exemplar-Based 3D Portrait Stylization

arXiv: https://arxiv.org/abs/2104.14559 · HTML: https://mlsanity.com/paper/2104.14559/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 45 references that are in the map.
- First posted on arXiv 29 Apr 2021.

## Builds on (most cited first, 21 of 21)

- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [Unsupervised Image-to-Image Translation Networks](https://mlsanity.com/paper/1703.00848/) (2017)
- [A Neural Algorithm of Artistic Style](https://mlsanity.com/paper/1508.06576/) (2015)
- [Neural 3D Mesh Renderer](https://mlsanity.com/paper/1711.07566/) (2017)
- [Universal Style Transfer via Feature Transforms](https://mlsanity.com/paper/1705.08086/) (2017)
- [DRIT++: Diverse Image-to-Image Translation via Disentangled Representations](https://mlsanity.com/paper/1905.01270/) (2019)
- [Texture Networks: Feed-forward Synthesis of Textures and Stylized Images](https://mlsanity.com/paper/1603.03417/) (2016)
- [Style Transfer by Relaxed Optimal Transport and Self-Similarity](https://mlsanity.com/paper/1904.12785/) (2019)
- [Visual Attribute Transfer through Deep Image Analogy](https://mlsanity.com/paper/1705.01088/) (2017)
- [Avatar-Net: Multi-scale Zero-shot Style Transfer by Feature Decoration](https://mlsanity.com/paper/1805.03857/) (2018)
- [Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis](https://mlsanity.com/paper/1601.04589/) (2016)
- [StyleBank: An Explicit Representation for Neural Image Style Transfer](https://mlsanity.com/paper/1703.09210/) (2017)
- [Arbitrary Style Transfer with Deep Feature Reshuffle](https://mlsanity.com/paper/1805.04103/) (2018)
- [WarpGAN: Automatic Caricature Generation](https://mlsanity.com/paper/1811.10100/) (2018)
- [Alive Caricature from 2D to 3D](https://mlsanity.com/paper/1803.06802/) (2018)
- [CaricatureShop: Personalized and Photorealistic Caricature Sketching](https://mlsanity.com/paper/1807.09064/) (2018)
- [CariGAN: Caricature Generation through Weakly Paired Adversarial Learning](https://mlsanity.com/paper/1811.00445/) (2018)
- [Learning to Caricature via Semantic Shape Transform](https://mlsanity.com/paper/2008.05090/) (2020)
- [Anime Style Space Exploration Using Metric Learning and Generative Adversarial Networks](https://mlsanity.com/paper/1805.07997/) (2018)

## Built on by (most cited first, 21 of 21)

- [Text2Mesh: Text-Driven Neural Stylization for Meshes](https://mlsanity.com/paper/2112.03221/) (2021)
- [NeRF-Art: Text-Driven Neural Radiance Fields Stylization](https://mlsanity.com/paper/2212.08070/) (2022)
- [AvatarCraft: Transforming Text into Neural Human Avatars with Parameterized Shape and Pose Control](https://mlsanity.com/paper/2303.17606/) (2023)
- [TANGO: Text-driven Photorealistic and Robust 3D Stylization via Lighting Decomposition](https://mlsanity.com/paper/2210.11277/) (2022)
- [StyleMesh: Style Transfer for Indoor 3D Scene Reconstructions](https://mlsanity.com/paper/2112.01530/) (2021)
- [3D Question Answering](https://mlsanity.com/paper/2112.08359/) (2021)
- [3DAvatarGAN: Bridging Domains for Personalized Editable Avatars](https://mlsanity.com/paper/2301.02700/) (2023)
- [Chat-Edit-3D: Interactive 3D Scene Editing via Text Prompts](https://mlsanity.com/paper/2407.06842/) (2024)
- [Creative Birds: Self-Supervised Single-View 3D Style Transfer](https://mlsanity.com/paper/2307.14127/) (2023)
- [Editing 3D Scenes via Text Prompts without Retraining](https://mlsanity.com/paper/2309.04917/) (2023)
- [HyperStyle3D: Text-Guided 3D Portrait Stylization via Hypernetworks](https://mlsanity.com/paper/2304.09463/) (2023)
- [Robust Point Cloud Segmentation with Noisy Annotations](https://mlsanity.com/paper/2212.03242/) (2022)
- [StyO: Stylize Your Face in Only One-shot](https://mlsanity.com/paper/2303.03231/) (2023)
- [Instruct-Video2Avatar: Video-to-Avatar Generation with Instructions](https://mlsanity.com/paper/2306.02903/) (2023)
- [AlteredAvatar: Stylizing Dynamic 3D Avatars with Fast Style Adaptation](https://mlsanity.com/paper/2305.19245/) (2023)
- [NARRATE: A Normal Assisted Free-View Portrait Stylizer](https://mlsanity.com/paper/2207.00974/) (2022)
- [Realtime Data-Efficient Portrait Stylization Based On Geometric Alignment](https://mlsanity.com/paper/2211.15549/) (2022)
- [Multi-Modal Face Stylization with a Generative Prior](https://mlsanity.com/paper/2305.18009/) (2023)
- [Chat-Edit-3D++: Interactive 3D and 4D Scene Editing via Large Language Models](https://mlsanity.com/paper/2608.29137/) (2026)
- [3D Face Arbitrary Style Transfer](https://mlsanity.com/paper/2303.07709/) (2023)
- [3Deformer: A Common Framework for Image-Guided Mesh Deformation](https://mlsanity.com/paper/2307.09892/) (2023)

## Abstract (arXiv)

Exemplar-based portrait stylization is widely attractive and highly desired. Despite recent successes, it remains challenging, especially when considering both texture and geometric styles. In this paper, we present the first framework for one-shot 3D portrait style transfer, which can generate 3D face models with both the geometry exaggerated and the texture stylized while preserving the identity from the original content. It requires only one arbitrary style image instead of a large set of training examples for a particular style, provides geometry and texture outputs that are fully parameterized and disentangled, and enables further graphics applications with the 3D representations. The framework consists of two stages. In the first geometric style transfer stage, we use facial landmark translation to capture the coarse geometry style and guide the deformation of the dense 3D face geometry. In the second texture style transfer stage, we focus on performing style transfer on the canonical texture by adopting a differentiable renderer to optimize the texture in a multi-view framework. Experiments show that our method achieves robustly good results on different artistic styles and outperforms existing methods. We also demonstrate the advantages of our method via various 2D and 3D graphics applications. Project page is https://halfjoe.github.io/projs/3DPS/index.html.