# NeRFInvertor: High Fidelity NeRF-GAN Inversion for Single-shot Real Image Animation

arXiv: https://arxiv.org/abs/2211.17235 · HTML: https://mlsanity.com/paper/2211.17235/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 52 references that are in the map.
- First posted on arXiv 30 Nov 2022.

## Builds on (most cited first, 30 of 38)

- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [Efficient Geometry-aware 3D Generative Adversarial Networks](https://mlsanity.com/paper/2112.07945/) (2021)
- [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020)
- [PlenOctrees for Real-time Rendering of Neural Radiance Fields](https://mlsanity.com/paper/2103.14024/) (2021)
- [Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation](https://mlsanity.com/paper/2008.00951/) (2020)
- [pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2012.00926/) (2020)
- [GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2007.02442/) (2020)
- [Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?](https://mlsanity.com/paper/1904.03189/) (2019)
- [Designing an Encoder for StyleGAN Image Manipulation](https://mlsanity.com/paper/2102.02766/) (2021)
- [Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Set](https://mlsanity.com/paper/1903.08527/) (2019)
- [One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing](https://mlsanity.com/paper/2011.15126/) (2020)
- [Pivotal Tuning for Latent-based Editing of Real Images](https://mlsanity.com/paper/2106.05744/) (2021)
- [Image2StyleGAN++: How to Edit the Embedded Images?](https://mlsanity.com/paper/1911.11544/) (2019)
- [StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows](https://mlsanity.com/paper/2008.02401/) (2020)
- [Towards Real-World Blind Face Restoration with Generative Facial Prior](https://mlsanity.com/paper/2101.04061/) (2021)
- [EMOCA: Emotion Driven Monocular Face Capture and Animation](https://mlsanity.com/paper/2204.11312/) (2022)
- [HeadNeRF: A Real-time NeRF-based Parametric Head Model](https://mlsanity.com/paper/2112.05637/) (2021)
- [GRAM: Generative Radiance Manifolds for 3D-Aware Image Generation](https://mlsanity.com/paper/2112.08867/) (2021)
- [Swapping Autoencoder for Deep Image Manipulation](https://mlsanity.com/paper/2007.00653/) (2020)
- [GAN Prior Embedded Network for Blind Face Restoration in the Wild](https://mlsanity.com/paper/2105.06070/) (2021)
- [Editing in Style: Uncovering the Local Semantics of GANs](https://mlsanity.com/paper/2004.14367/) (2020)
- [Fourier PlenOctrees for Dynamic Radiance Field Rendering in Real-time](https://mlsanity.com/paper/2202.08614/) (2022)
- [LOLNeRF: Learn from One Look](https://mlsanity.com/paper/2111.09996/) (2021)
- [Pix2NeRF: Unsupervised Conditional π-GAN for Single Image to Neural Radiance Fields Translation](https://mlsanity.com/paper/2202.13162/) (2022)
- [3D GAN Inversion for Controllable Portrait Image Animation](https://mlsanity.com/paper/2203.13441/) (2022)
- [High-Fidelity 3D Digital Human Head Creation from RGB-D Selfies](https://mlsanity.com/paper/2010.05562/) (2020)
- [AniFaceGAN: Animatable 3D-Aware Face Image Generation for Video Avatars](https://mlsanity.com/paper/2210.06465/) (2022)

## Built on by (most cited first, 23 of 23)

- [Portrait4D: Learning One-Shot 4D Head Avatar Synthesis using Synthetic Data](https://mlsanity.com/paper/2311.18729/) (2023)
- [HeadGAP: Few-Shot 3D Head Avatar via Generalizable Gaussian Priors](https://mlsanity.com/paper/2408.06019/) (2024)
- [VOODOO 3D: Volumetric Portrait Disentanglement for One-Shot 3D Head Reenactment](https://mlsanity.com/paper/2312.04651/) (2023)
- [Real-time 3D-aware Portrait Video Relighting](https://mlsanity.com/paper/2410.18355/) (2024)
- [BARD-GS: Blur-Aware Reconstruction of Dynamic Scenes via Gaussian Splatting](https://mlsanity.com/paper/2503.15835/) (2025)
- [DecoupledGaussian: Object-Scene Decoupling for Physics-Based Interaction](https://mlsanity.com/paper/2503.05484/) (2025)
- [UltrAvatar: A Realistic Animatable 3D Avatar Diffusion Model with Authenticity Guided Textures](https://mlsanity.com/paper/2401.11078/) (2024)
- [Neural Radiance Field-based Visual Rendering: A Comprehensive Review](https://mlsanity.com/paper/2404.00714/) (2024)
- [Diffusion-driven GAN Inversion for Multi-Modal Face Image Generation](https://mlsanity.com/paper/2405.04356/) (2024)
- [Semantically-aware Neural Radiance Fields for Visual Scene Understanding: A Comprehensive Review](https://mlsanity.com/paper/2402.11141/) (2024)
- [View-consistent Object Removal in Radiance Fields](https://mlsanity.com/paper/2408.02100/) (2024)
- [Advancing Talking Head Generation: A Comprehensive Survey of Multi-Modal Methodologies, Datasets, Evaluation Metrics, and Loss Functions](https://mlsanity.com/paper/2507.02900/) (2025)
- [LatentSwap3D: Semantic Edits on 3D Image GANs](https://mlsanity.com/paper/2212.01381/) (2022)
- [Dynamic NeRF: A Review](https://mlsanity.com/paper/2405.08609/) (2024)
- [INFAMOUS-NeRF: ImproviNg FAce MOdeling Using Semantically-Aligned Hypernetworks with Neural Radiance Fields](https://mlsanity.com/paper/2312.16197/) (2023)
- [G-NeRF: Geometry-enhanced Novel View Synthesis from Single-View Images](https://mlsanity.com/paper/2404.07474/) (2024)
- [MomentsNeRF: Leveraging Orthogonal Moments for Few-Shot Neural Rendering](https://mlsanity.com/paper/2407.02668/) (2024)
- [G3FA: Geometry-guided GAN for Face Animation](https://mlsanity.com/paper/2408.13049/) (2024)
- [Learning and Evaluating Human Preferences for Conversational Head Generation](https://mlsanity.com/paper/2307.10636/) (2023)
- [Self-supervised Learning for Enhancing Geometrical Modeling in 3D-Aware Generative Adversarial Network](https://mlsanity.com/paper/2312.11856/) (2023)
- [ZIGNeRF: Zero-shot 3D Scene Representation with Invertible Generative Neural Radiance Fields](https://mlsanity.com/paper/2306.02741/) (2023)
- [3DProxyImg: Controllable 3D-Aware Animation Synthesis from Single Image via 2D-3D Aligned Proxy Embedding](https://mlsanity.com/paper/2512.15126/) (2025)
- [SpatialAvatar-0: High-Quality 4D Head Avatar with Multi-Stage Reconstruction](https://mlsanity.com/paper/2606.15659/) (2026)

## Abstract (arXiv)

Nerf-based Generative models have shown impressive capacity in generating high-quality images with consistent 3D geometry. Despite successful synthesis of fake identity images randomly sampled from latent space, adopting these models for generating face images of real subjects is still a challenging task due to its so-called inversion issue. In this paper, we propose a universal method to surgically fine-tune these NeRF-GAN models in order to achieve high-fidelity animation of real subjects only by a single image. Given the optimized latent code for an out-of-domain real image, we employ 2D loss functions on the rendered image to reduce the identity gap. Furthermore, our method leverages explicit and implicit 3D regularizations using the in-domain neighborhood samples around the optimized latent code to remove geometrical and visual artifacts. Our experiments confirm the effectiveness of our method in realistic, high-fidelity, and 3D consistent animation of real faces on multiple NeRF-GAN models across different datasets.