mlsanityData to 14 Oct 2026

High-fidelity 3D GAN Inversion by Pseudo-multi-view Optimization

arXiv 2211.15662 ↗First posted 28 Nov 2022cs.CVcited by 65

Not on the map. Why?

Abstract (arXiv)

We present a high-fidelity 3D generative adversarial network (GAN) inversion framework that can synthesize photo-realistic novel views while preserving specific details of the input image. High-fidelity 3D GAN inversion is inherently challenging due to the geometry-texture trade-off in 3D inversion, where overfitting to a single view input image often damages the estimated geometry during the latent optimization. To solve this challenge, we propose a novel pipeline that builds on the pseudo-multi-view estimation with visibility analysis. We keep the original textures for the visible parts and utilize generative priors for the occluded parts. Extensive experiments show that our approach achieves advantageous reconstruction and novel view synthesis quality over state-of-the-art methods, even for images with out-of-distribution textures. The proposed pipeline also enables image attribute editing with the inverted latent code and 3D-aware texture modification. Our approach enables high-fidelity 3D rendering from a single image, which is promising for various applications of AI-generated 3D content.

Key facts. Cited by 65 papers in the map (data through 14 Oct 2026). Builds on 74 of its 84 references that are in the map. First posted on arXiv 28 Nov 2022.

Builds on

74 of 84 references.

References · 74

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  4. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  5. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  6. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  7. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  8. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
  9. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  10. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,098
  11. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  12. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
38 more
  1. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 785
  2. 2020Convolutional Occupancy Networkscited by 761
  3. 2020Encoding in Style: a StyleGAN Encoder for Image-to-Image Translationcited by 727
  4. 2020Implicit Geometric Regularization for Learning Shapescited by 710
  5. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  6. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 703
  7. 2019Interpreting the Latent Space of GANs for Semantic Face Editingcited by 695
  8. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  9. 2019MaskGAN: Towards Diverse and Interactive Facial Image Manipulationcited by 628
  10. 2018Free-Form Image Inpainting with Gated Convolutioncited by 594
  11. 2021Designing an Encoder for StyleGAN Image Manipulationcited by 544
  12. 2019Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Setcited by 482
  13. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  14. 2021Pivotal Tuning for Latent-based Editing of Real Imagescited by 427
  15. 2020In-Domain GAN Inversion for Real Image Editingcited by 420
  16. 2019SAL: Sign Agnostic Learning of Shapes from Raw Datacited by 390
  17. 2021GAN Inversion: A Surveycited by 367
  18. 2019Image2StyleGAN++: How to Edit the Embedded Images?cited by 356
  19. 2016Generative Visual Manipulation on the Natural Image Manifoldcited by 348
  20. 2020StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generationcited by 332
  21. 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
  22. 2021StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generationcited by 289
  23. 2021ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinementcited by 248
  24. 2020Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulationcited by 225
  25. 2017Photographic Image Synthesis with Cascaded Refinement Networkscited by 223
  26. 2021Large Scale Image Completion via Co-Modulated Generative Adversarial Networkscited by 220
  27. 2021GRAM: Generative Radiance Manifolds for 3D-Aware Image Generationcited by 216
  28. 2021HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editingcited by 208
  29. 2021High-Fidelity GAN Inversion for Image Attribute Editingcited by 197
  30. 2019Image Processing Using Multi-Code GAN Priorcited by 185
  31. 2020Editing in Style: Uncovering the Local Semantics of GANscited by 183
  32. 2016Learning Shape Abstractions by Assembling Volumetric Primitivescited by 181
  33. 2018Escaping Plato's Cave: 3D Shape From Adversarial Renderingcited by 170
  34. 2020BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Imagescited by 166
  35. 2021CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesiscited by 149
  36. 2016Inverting The Generator Of A Generative Adversarial Networkcited by 140
  37. 2016Invertible Conditional GANs for image editingcited by 118
  38. 2019Deep Flow-Guided Video Inpaintingcited by 116

50 most cited of 74 All 74 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 34; 2024: 21; 2025: 8; 2026: 2.

65 papers.

Sort:

By area:

  1. 2023HexPlane: A Fast Representation for Dynamic Scenescited by 587
  2. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 343
  3. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Modelscited by 274
  4. 2024Generalizable and Animatable Gaussian Head Avatarcited by 97
  5. 2023Real-Time Radiance Fields for Single-Image Portrait View Synthesiscited by 64
  6. 2023Consistent123: One Image to Highly Consistent 3D Asset Using Case-Aware Diffusion Priorscited by 64
  7. 2023Portrait4D: Learning One-Shot 4D Head Avatar Synthesis using Synthetic Datacited by 55
  8. 2024HeadGAP: Few-Shot 3D Head Avatar via Generalizable Gaussian Priorscited by 44
  9. 2023TriPlaneNet: An Encoder for EG3D Inversioncited by 41
  10. 2023Make Encoder Great Again in 3D GAN Inversion through Geometry and Occlusion-Aware Encodingcited by 35
  11. 2023VOODOO 3D: Volumetric Portrait Disentanglement for One-Shot 3D Head Reenactmentcited by 33
  12. 2024Holo-Relighting: Controllable Volumetric Portrait Relighting from a Single Imagecited by 32
38 more
  1. 2023Blind Video Deflickering by Neural Filtering with a Flawed Atlascited by 27
  2. 2024Lite2Relight: 3D-aware Single Image Portrait Relightingcited by 25
  3. 2023LPFF: A Portrait Dataset for Face Generators Across Large Posescited by 22
  4. 2023Gaussian3Diff: 3D Gaussian Diffusion for 3D Full Head Synthesis and Editingcited by 22
  5. 2023Generalizable One-shot Neural Head Avatarcited by 21
  6. 2024Real-time 3D-aware Portrait Video Relightingcited by 21
  7. 2024Order Matters: Exploring Order Sensitivity in Multimodal Large Language Modelscited by 20
  8. 2023InvertAvatar: Incremental GAN Inversion for Generalized Head Avatarscited by 19
  9. 2023Single-Shot Implicit Morphable Faces with Consistent Texture Parameterizationcited by 16
  10. 2024DaReNeRF: Direction-aware Representation for Dynamic Scenescited by 15
  11. 2024Learning to Generate Conditional Tri-plane for 3D-aware Expression Controllable Portrait Animationcited by 15
  12. 2024Portrait3D: Text-Guided High-Quality 3D Portrait Generation Using Pyramid Representation and GANs Priorcited by 15
  13. 2023Learning 3D-aware Image Synthesis with Unknown Pose Distributioncited by 14
  14. 2023DiFaReli++: Diffusion Face Relighting with Consistent Cast Shadowscited by 14
  15. 20233D-aware Blending with Generative NeRFscited by 13
  16. 2024Isotropic3D: Image-to-3D Generation Based on a Single CLIP Embeddingcited by 13
  17. 2023Ray Conditioning: Trading Photo-consistency for Photo-realism in Multi-view Image Generationcited by 11
  18. 2023InstructPix2NeRF: Instructed 3D Portrait Editing from a Single Imagecited by 10
  19. 2024Real-time 3D-aware Portrait Editing from a Single Imagecited by 10
  20. 2023Controllable 3D Face Generation with Conditional Style Code Diffusioncited by 9
  21. 2024Reference-Based 3D-Aware Image Editing with Triplanescited by 9
  22. 2023My3DGen: A Scalable Personalized 3D Generative Modelcited by 8
  23. 20233DPortraitGAN: Learning One-Quarter Headshot 3D GANs from a Single-View Portrait Dataset with Diverse Body Posescited by 8
  24. 2023WildFusion: Learning 3D-Aware Latent Diffusion Models in View Spacecited by 8
  25. 2023DisCO: Portrait Distortion Correction with Perspective-Aware 3D GANscited by 6
  26. 2023Warping the Residuals for Image Editing with StyleGANcited by 6
  27. 2025VRM: Knowledge Distillation via Virtual Relation Matchingcited by 6
  28. 2025PercHead: Perceptual Head Model for Single-Image 3D Head Reconstruction & Editingcited by 6
  29. 2023High-Quality 3D Creation from A Single Image Using Subject-Specific Knowledge Priorcited by 5
  30. 2024Dual Encoder GAN Inversion for High-Fidelity 3D Head Reconstruction from Single Imagescited by 5
  31. 2023In-N-Out: Faithful 3D GAN Inversion with Volumetric Decomposition for Face Editingcited by 4
  32. 2023TEGLO: High Fidelity Canonical Texture Mapping from Single-View Imagescited by 4
  33. 20233D-GOI: 3D GAN Omni-Inversion for Multifaceted and Multi-object Editingcited by 4
  34. 2025Cross-Architecture Distillation Made Simple with Redundancy Suppressioncited by 4
  35. 2023Multi-view Inversion for 3D-aware Generative Adversarial Networkscited by 2
  36. 2024Talk3D: High-Fidelity Talking Portrait Synthesis via Personalized 3D Generative Priorcited by 2
  37. 2024FaceLift: Semi-supervised 3D Facial Landmark Localizationcited by 2
  38. 2024Identity Preserving 3D Head Stylization with Multiview Score Distillationcited by 2

50 most cited of 65 All 65 (JSON)