# TriPlaneNet: An Encoder for EG3D Inversion

arXiv: https://arxiv.org/abs/2303.13497 · HTML: https://mlsanity.com/paper/2303.13497/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 59 of its 77 references that are in the map.
- First posted on arXiv 23 Mar 2023.

## Builds on (most cited first, 30 of 59)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Efficient Geometry-aware 3D Generative Adversarial Networks](https://mlsanity.com/paper/2112.07945/) (2021)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://mlsanity.com/paper/1606.03657/) (2016)
- [GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields](https://mlsanity.com/paper/2011.12100/) (2020)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation](https://mlsanity.com/paper/2008.00951/) (2020)
- [pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2012.00926/) (2020)
- [GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2007.02442/) (2020)
- [Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?](https://mlsanity.com/paper/1904.03189/) (2019)
- [GANSpace: Discovering Interpretable GAN Controls](https://mlsanity.com/paper/2004.02546/) (2020)
- [LION: Latent Point Diffusion Models for 3D Shape Generation](https://mlsanity.com/paper/2210.06978/) (2022)
- [Designing an Encoder for StyleGAN Image Manipulation](https://mlsanity.com/paper/2102.02766/) (2021)
- [Label-Efficient Semantic Segmentation with Diffusion Models](https://mlsanity.com/paper/2112.03126/) (2021)
- [StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis](https://mlsanity.com/paper/2110.08985/) (2021)
- [Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstruction](https://mlsanity.com/paper/2012.03065/) (2020)
- [Pivotal Tuning for Latent-based Editing of Real Images](https://mlsanity.com/paper/2106.05744/) (2021)
- [In-Domain GAN Inversion for Real Image Editing](https://mlsanity.com/paper/2004.00049/) (2020)
- [DeepVoxels: Learning Persistent 3D Feature Embeddings](https://mlsanity.com/paper/1812.01024/) (2018)
- [GAN Inversion: A Survey](https://mlsanity.com/paper/2101.05278/) (2021)
- [Image2StyleGAN++: How to Edit the Embedded Images?](https://mlsanity.com/paper/1911.11544/) (2019)

## Built on by (most cited first, 30 of 41)

- [AGG: Amortized Generative 3D Gaussians for Single Image to 3D](https://mlsanity.com/paper/2401.04099/) (2024)
- [Avat3r: Large Animatable Gaussian Reconstruction Model for High-fidelity 3D Head Avatars](https://mlsanity.com/paper/2502.20220/) (2025)
- [VOODOO 3D: Volumetric Portrait Disentanglement for One-Shot 3D Head Reenactment](https://mlsanity.com/paper/2312.04651/) (2023)
- [Holo-Relighting: Controllable Volumetric Portrait Relighting from a Single Image](https://mlsanity.com/paper/2403.09632/) (2024)
- [DiffPortrait3D: Controllable Diffusion for Zero-Shot Portrait View Synthesis](https://mlsanity.com/paper/2312.13016/) (2023)
- [FaceLift: Learning Generalizable Single Image 3D Face Reconstruction from Synthetic Heads](https://mlsanity.com/paper/2412.17812/) (2024)
- [Generalizable One-shot Neural Head Avatar](https://mlsanity.com/paper/2306.08768/) (2023)
- [Real-time 3D-aware Portrait Video Relighting](https://mlsanity.com/paper/2410.18355/) (2024)
- [InvertAvatar: Incremental GAN Inversion for Generalized Head Avatars](https://mlsanity.com/paper/2312.02222/) (2023)
- [Learning to Generate Conditional Tri-plane for 3D-aware Expression Controllable Portrait Animation](https://mlsanity.com/paper/2404.00636/) (2024)
- [MVPortrait: Text-Guided Motion and Emotion Control for Multi-view Vivid Portrait Animation](https://mlsanity.com/paper/2503.19383/) (2025)
- [A Survey on 3D Human Avatar Modeling -- From Reconstruction to Generation](https://mlsanity.com/paper/2406.04253/) (2024)
- [Reference-Based 3D-Aware Image Editing with Triplanes](https://mlsanity.com/paper/2404.03632/) (2024)
- [My3DGen: A Scalable Personalized 3D Generative Model](https://mlsanity.com/paper/2307.05468/) (2023)
- [PanoLAM: Large Avatar Model for Gaussian Full-Head Synthesis from One-shot Unposed Image](https://mlsanity.com/paper/2509.07552/) (2025)
- [DiffAge3D: Diffusion-based 3D-aware Face Aging](https://mlsanity.com/paper/2408.15922/) (2024)
- [PercHead: Perceptual Head Model for Single-Image 3D Head Reconstruction & Editing](https://mlsanity.com/paper/2511.02777/) (2025)
- [6Img-to-3D: Few-Image Large-Scale Outdoor Driving Scene Reconstruction](https://mlsanity.com/paper/2404.12378/) (2024)
- [SYM3D: Learning Symmetric Triplanes for Better 3D-Awareness of GANs](https://mlsanity.com/paper/2406.06432/) (2024)
- [Dual Encoder GAN Inversion for High-Fidelity 3D Head Reconstruction from Single Images](https://mlsanity.com/paper/2409.20530/) (2024)
- [3D-GOI: 3D GAN Omni-Inversion for Multifaceted and Multi-object Editing](https://mlsanity.com/paper/2311.12050/) (2023)
- [Coherent 3D Portrait Video Reconstruction via Triplane Fusion](https://mlsanity.com/paper/2405.00794/) (2024)
- [VoluMe -- Authentic 3D Video Calls from Live Gaussian Splat Prediction](https://mlsanity.com/paper/2507.21311/) (2025)
- [Multi-view Inversion for 3D-aware Generative Adversarial Networks](https://mlsanity.com/paper/2312.05330/) (2023)
- [Talk3D: High-Fidelity Talking Portrait Synthesis via Personalized 3D Generative Prior](https://mlsanity.com/paper/2403.20153/) (2024)
- [Identity Preserving 3D Head Stylization with Multiview Score Distillation](https://mlsanity.com/paper/2411.13536/) (2024)
- [FlexAvatar: Flexible Large Reconstruction Model for Animatable Gaussian Head Avatars with Detailed Deformation](https://mlsanity.com/paper/2512.17717/) (2025)
- [SoPE: Spherical Coordinate-Based Positional Embedding for Enhancing Spatial Perception of 3D LVLMs](https://mlsanity.com/paper/2602.22716/) (2026)
- [SUPER: Selfie Undistortion and Head Pose Editing with Identity Preservation](https://mlsanity.com/paper/2406.12700/) (2024)
- [From Blurry to Believable: Enhancing Low-quality Talking Heads with 3D Generative Priors](https://mlsanity.com/paper/2602.06122/) (2026)

## Abstract (arXiv)

Recent progress in NeRF-based GANs has introduced a number of approaches for high-resolution and high-fidelity generative modeling of human heads with a possibility for novel view rendering. At the same time, one must solve an inverse problem to be able to re-render or modify an existing image or video. Despite the success of universal optimization-based methods for 2D GAN inversion, those applied to 3D GANs may fail to extrapolate the result onto the novel view, whereas optimization-based 3D GAN inversion methods are time-consuming and can require at least several minutes per image. Fast encoder-based techniques, such as those developed for StyleGAN, may also be less appealing due to the lack of identity preservation. Our work introduces a fast technique that bridges the gap between the two approaches by directly utilizing the tri-plane representation presented for the EG3D generative model. In particular, we build upon a feed-forward convolutional encoder for the latent code and extend it with a fully-convolutional predictor of tri-plane numerical offsets. The renderings are similar in quality to the ones produced by optimization-based techniques and outperform the ones by encoder-based methods. As we empirically prove, this is a consequence of directly operating in the tri-plane space, not in the GAN parameter space, while making use of an encoder-based trainable approach. Finally, we demonstrate significantly more correct embedding of a face image in 3D than for all the baselines, further strengthened by a probably symmetric prior enabled during training.