mlsanityData to 14 Oct 2026

3D-GOI: 3D GAN Omni-Inversion for Multifaceted and Multi-object Editing

arXiv 2311.12050 ↗First posted 18 Nov 2023cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

The current GAN inversion methods typically can only edit the appearance and shape of a single object and background while overlooking spatial information. In this work, we propose a 3D editing framework, 3D-GOI, to enable multifaceted editing of affine information (scale, translation, and rotation) on multiple objects. 3D-GOI realizes the complex editing function by inverting the abundance of attribute codes (object shape/appearance/scale/rotation/translation, background shape/appearance, and camera pose) controlled by GIRAFFE, a renowned 3D GAN. Accurately inverting all the codes is challenging, 3D-GOI solves this challenge following three main steps. First, we segment the objects and the background in a multi-object image. Second, we use a custom Neural Inversion Encoder to obtain coarse codes of each object. Finally, we use a round-robin optimization algorithm to get precise codes to reconstruct the image. To the best of our knowledge, 3D-GOI is the first framework to enable multifaceted editing on multiple objects. Both qualitative and quantitative experiments demonstrate that 3D-GOI holds immense potential for flexible, multifaceted editing in complex multi-object scenes.Our project and code are released at https://3d-goi.github.io .

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 39 of its 45 references that are in the map. First posted on arXiv 18 Nov 2023.

Builds on

39 of 45 references.

References · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  6. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  7. 2022Generative Adversarial Networkscited by 5,627
  8. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  9. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  10. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  11. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  12. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
27 more
  1. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  2. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 785
  3. 2020Encoding in Style: a StyleGAN Encoder for Image-to-Image Translationcited by 727
  4. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 689
  5. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  6. 2021Designing an Encoder for StyleGAN Image Manipulationcited by 544
  7. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  8. 2021Pivotal Tuning for Latent-based Editing of Real Imagescited by 427
  9. 2020In-Domain GAN Inversion for Real Image Editingcited by 420
  10. 2016Generative Visual Manipulation on the Natural Image Manifoldcited by 348
  11. 2021HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editingcited by 208
  12. 2021High-Fidelity GAN Inversion for Image Attribute Editingcited by 197
  13. 2020BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Imagescited by 166
  14. 2019Seeing What a GAN Cannot Generatecited by 148
  15. 2016Invertible Conditional GANs for image editingcited by 118
  16. 2020Transforming and Projecting Images into Class-conditional Generative Networkscited by 76
  17. 2022High-fidelity 3D GAN Inversion by Pseudo-multi-view Optimizationcited by 65
  18. 2019HoloGAN: Unsupervised learning of 3D representations from natural imagescited by 59
  19. 2023CompoNeRF: Text-guided Multi-object Compositional NeRF with Editable 3D Scene Layoutcited by 57
  20. 20223D GAN Inversion with Pose Optimizationcited by 44
  21. 2023TriPlaneNet: An Encoder for EG3D Inversioncited by 41
  22. 20223D GAN Inversion with Facial Symmetry Priorcited by 39
  23. 2021E2Style: Improve the Efficiency and Effectiveness of StyleGAN Inversioncited by 33
  24. 2021Scene Synthesis via Uncertainty-Driven Attribute Synchronizationcited by 29
  25. 2021Compositional Transformers for Scene Generationcited by 24
  26. 2022Self-Supervised Geometry-Aware Encoder for Style-Based 3D GAN Inversioncited by 24
  27. 2022Learning Detailed Radiance Manifolds for High-Fidelity and 3D-Consistent Portrait Synthesis from Monocular Imagecited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 1.

4 papers.

  1. 2024Taming Diffusion Prior for Image Super-Resolution with Domain Shift SDEscited by 13
  2. 2024Text-to-3D Generation by 2D Editingcited by 3
  3. 2025DreamScene: 3D Gaussian-based End-to-end Text-to-3D Scene Generationcited by 1
  4. 2026Hallo4D: Multi-Modal Hallucination Mitigation for Consistent Spatio-Temporal Generationcited by 0