3D-GIF: 3D-Controllable Object Generation via Implicit Factorized Representations
Not on the map. Why?
Abstract (arXiv)
While NeRF-based 3D-aware image generation methods enable viewpoint control, limitations still remain to be adopted to various 3D applications. Due to their view-dependent and light-entangled volume representation, the 3D geometry presents unrealistic quality and the color should be re-rendered for every desired viewpoint. To broaden the 3D applicability from 3D-aware image generation to 3D-controllable object generation, we propose the factorized representations which are view-independent and light-disentangled, and training schemes with randomly sampled light conditions. We demonstrate the superiority of our method by visualizing factorized representations, re-lighted images, and albedo-textured meshes. In addition, we show that our approach improves the quality of the generated geometry via visualization and quantitative comparison. To the best of our knowledge, this is the first work that extracts albedo-textured meshes with unposed 2D images without any additional labels or assumptions.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 43 of its 54 references that are in the map. First posted on arXiv 12 Mar 2022.
Builds on
43 of 54 references.
References · 43
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
- 2014Deep Learning Face Attributes in the Wildcited by 3,679
- 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
- 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
- 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
- 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
- 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
- 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
- 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
- 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
- 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
31 more
- 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
- 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
- 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
- 2021Depth-supervised NeRF: Fewer Views and Faster Training for Freecited by 732
- 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
- 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
- 2018Which Training Methods for GANs do actually Converge?cited by 672
- 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 649
- 2020Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humanscited by 634
- 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
- 2020NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesiscited by 493
- 2020NeRD: Neural Reflectance Decomposition from Image Collectionscited by 446
- 2018An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solutioncited by 302
- 2020StyleRig: Rigging StyleGAN for 3D Control over Portrait Imagescited by 267
- 2019Learning Shape Templates with Structured Implicit Functionscited by 242
- 2020PaMIR: Parametric Model-Conditioned Implicit Representation for Image-based Human Reconstructioncited by 231
- 2020Disentangled and Controllable Face Image Generation via 3D Imitative-Contrastive Learningcited by 219
- 2019Texture Fields: Learning Texture Representations in Function Spacecited by 210
- 20193D Point Cloud Generative Adversarial Network Based on Tree Structured Graph Convolutionscited by 173
- 2018Escaping Plato's Cave: 3D Shape From Adversarial Renderingcited by 170
- 2020BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Imagescited by 166
- 2021NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illuminationcited by 135
- 2018High Quality Monocular Depth Estimation via Transfer Learningcited by 133
- 20163D Shape Induction from 2D Views of Multiple Objectscited by 121
- 2020Geo-PIFu: Geometry and Pixel Aligned Implicit Functions for Single-view Human Reconstructioncited by 112
- 2019Learning to Infer Implicit Surfaces without 3D Supervisioncited by 103
- 2021A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesiscited by 72
- 2020Leveraging 2D Data to Learn Textured 3D Mesh Generationcited by 69
- 2020Lifting 2D StyleGAN for 3D-Aware Face Generationcited by 64
- 2020Convolutional Generation of Textured 3D Meshescited by 52
- 2020Progressive Point Cloud Deconvolution Generation Networkcited by 46
Built on by
Citing papers per year: 2022: 1; 2023: 2.
3 papers.