FaceRefiner: High-Fidelity Facial Texture Refinement with Differentiable Rendering-based Style Transfer
Not on the map. Why?
Abstract (arXiv)
Recent facial texture generation methods prefer to use deep networks to synthesize image content and then fill in the UV map, thus generating a compelling full texture from a single image. Nevertheless, the synthesized texture UV map usually comes from a space constructed by the training data or the 2D face generator, which limits the methods' generalization ability for in-the-wild input images. Consequently, their facial details, structures and identity may not be consistent with the input. In this paper, we address this issue by proposing a style transfer-based facial texture refinement method named FaceRefiner. FaceRefiner treats the 3D sampled texture as style and the output of a texture generation method as content. The photo-realistic style is then expected to be transferred from the style image to the content image. Different from current style transfer methods that only transfer high and middle level information to the result, our style transfer method integrates differentiable rendering to also transfer low level (or pixel level) information in the visible face regions. The main benefit of such multi-level information transfer is that, the details, structures and semantics in the input can thus be well preserved. The extensive experiments on Multi-PIE, CelebA and FFHQ datasets demonstrate that our refinement method can improve the texture quality and the face identity preserving ability, compared with state-of-the-arts.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 39 of its 59 references that are in the map. First posted on arXiv 8 Jan 2026.
Builds on
39 of 59 references.
References · 39
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
- 2014Deep Learning Face Attributes in the Wildcited by 3,679
- 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
- 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
- 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
- 2018Free-Form Image Inpainting with Gated Convolutioncited by 594
- 2018Generative Image Inpainting with Contextual Attentioncited by 578
- 2018Image Inpainting for Irregular Holes Using Partial Convolutionscited by 505
- 2019Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Setcited by 481
- 2020Modular Primitives for High-Performance Differentiable Renderingcited by 439
- 2017Universal Style Transfer via Feature Transformscited by 352
27 more
- 2021ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinementcited by 248
- 2021StyTr2: Image Style Transfer with Transformerscited by 186
- 2019Style Transfer by Relaxed Optimal Transport and Self-Similaritycited by 167
- 2019Pluralistic Image Completioncited by 162
- 2019GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstructioncited by 150
- 2014Hypercolumns for Object Segmentation and Fine-grained Localizationcited by 128
- 2015A Light CNN for Deep Face Representation with Noisy Labelscited by 126
- 2021ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flowscited by 113
- 2016Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesiscited by 102
- 2020AvatarMe: Realistically Renderable 3D Facial Reconstruction "in-the-wild"cited by 91
- 2020Arbitrary Video Style Transfer via Multi-Channel Correlationcited by 84
- 2017UV-GAN: Adversarial Facial UV Map Completion for Pose-invariant Face Recognitioncited by 62
- 2018A Style-Aware Content Loss for Real-time HD Style Transfercited by 58
- 2020Towards High-Fidelity 3D Face Reconstruction from In-the-Wild Images Using Graph Convolutional Networkscited by 51
- 2019Combining 3D Morphable Models: A Large scale Face-and-Head Modelcited by 49
- 20173D Face Morphable Models "In-the-Wild"cited by 44
- 2020Learning to Incorporate Structure Knowledge for Image Inpaintingcited by 37
- 2020OSTeC: One-Shot Texture Completioncited by 37
- 2014Effective Face Frontalization in Unconstrained Imagescited by 34
- 2020Deformable Style Transfercited by 31
- 2021In the light of feature distributions: moment matching for Neural Style Transfercited by 26
- 2016Semantic Style Transfer and Turning Two-Bit Doodles into Fine Artworkscited by 22
- 2014Feedforward semantic segmentation with zoom-out featurescited by 21
- 2021MeInGame: Create a Game Character Face from a Single Portraitcited by 16
- 2023EMEF: Ensemble Multi-Exposure Image Fusioncited by 10
- 2019Multi-Prototype Networks for Unconstrained Set-based Face Recognitioncited by 6
- 2024DNPM: A Neural Parametric Model for the Synthesis of Facial Geometric Detailscited by 4
Built on by
Citing papers per year: 2026: 3.
3 papers.