mlsanityData to 7 Oct 2026

Generative Deformable Radiance Fields for Disentangled Image Synthesis of Topology-Varying Objects

arXiv 2209.04183 ↗First posted 9 Sep 2022cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

3D-aware generative models have demonstrated their superb performance to generate 3D neural radiance fields (NeRF) from a collection of monocular 2D images even for topology-varying object categories. However, these methods still lack the capability to separately control the shape and appearance of the objects in the generated radiance fields. In this paper, we propose a generative model for synthesizing radiance fields of topology-varying objects with disentangled shape and appearance variations. Our method generates deformable radiance fields, which builds the dense correspondence between the density fields of the objects and encodes their appearances in a shared template field. Our disentanglement is achieved in an unsupervised manner without introducing extra labels to previous 3D-aware GAN training. We also develop an effective image inversion scheme for reconstructing the radiance field of an object in a real monocular image and manipulating its shape and appearance. Experiments show that our method can successfully learn the generative model from unstructured monocular images and well disentangle the shape and appearance for objects (e.g., chairs) with large topological variance. The model trained on synthetic data can faithfully reconstruct the real object in a given single image and achieve high-quality texture and shape editing results.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 53 of its 71 references that are in the map. First posted on arXiv 9 Sep 2022.

Builds on

53 of 71 references.

References · 53

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  4. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  5. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  6. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  7. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  8. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  9. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  10. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  11. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  12. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,013
38 more
  1. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  2. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
  3. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,607
  4. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,295
  5. 2018Demystifying MMD GANscited by 1,268
  6. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  7. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  8. 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 858
  9. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  10. 2020Convolutional Occupancy Networkscited by 760
  11. 2022Block-NeRF: Scalable Large Scene Neural View Synthesiscited by 720
  12. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  13. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  14. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  15. 2018Which Training Methods for GANs do actually Converge?cited by 672
  16. 2020Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humanscited by 634
  17. 2018AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generationcited by 539
  18. 2020In-Domain GAN Inversion for Real Image Editingcited by 420
  19. 2018DeepVoxels: Learning Persistent 3D Feature Embeddingscited by 401
  20. 2021CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fieldscited by 369
  21. 2020StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flowscited by 348
  22. 20163D Menagerie: Modeling the 3D shape and pose of animalscited by 310
  23. 2021Mixture of Volumetric Primitives for Efficient Neural Renderingcited by 277
  24. 2021Editing Conditional Radiance Fieldscited by 222
  25. 2020Disentangled and Controllable Face Image Generation via 3D Imitative-Contrastive Learningcited by 219
  26. 2021GRAM: Generative Radiance Manifolds for 3D-Aware Image Generationcited by 216
  27. 2021CodeNeRF: Disentangled Neural Radiance Fields for Object Categoriescited by 183
  28. 2018GANimation: Anatomically-aware Facial Animation from a Single Imagecited by 160
  29. 2021Unconstrained Scene Generation with Locally Conditioned Radiance Fieldscited by 130
  30. 2015Deep Convolutional Inverse Graphics Networkcited by 121
  31. 2020Deformed Implicit Field: Modeling 3D Shapes with Learned Dense Correspondencecited by 119
  32. 2020i3DMM: Deep Implicit 3D Morphable Model of Human Headscited by 109
  33. 2021NeuTex: Neural Texture Mapping for Volumetric Neural Renderingcited by 98
  34. 2020Deep Implicit Templates for 3D Shape Representationcited by 86
  35. 2022ImFace: A Nonlinear 3D Morphable Face Model with Implicit Neural Representationscited by 64
  36. 2015Multi-view 3D Models from Single Images with a Convolutional Networkcited by 57
  37. 2021FiG-NeRF: Figure-Ground Neural Radiance Fields for 3D Object Category Modellingcited by 47
  38. 2022Disentangled3D: Learning a 3D Generative Model with Disentangled Geometry and Appearance from Monocular Imagescited by 39

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 5.

8 papers.

  1. 2023Neural Residual Radiance Fields for Streamably Free-Viewpoint Videoscited by 77
  2. 2022AniFaceGAN: Animatable 3D-Aware Face Image Generation for Video Avatarscited by 46
  3. 2022HumanGen: Generating Human Radiance Fields with Explicit Priorscited by 33
  4. 2023NaviNeRF: NeRF-based 3D Representation Disentanglement by Latent Semantic Navigationcited by 8
  5. 2023textttNePhi: Neural Deformation Fields for Approximately Diffeomorphic Medical Image Registrationcited by 8
  6. 2022Learning Detailed Radiance Manifolds for High-Fidelity and 3D-Consistent Portrait Synthesis from Monocular Imagecited by 7
  7. 2023Fashion-model pose recommendation and generation using Machine Learningcited by 0
  8. 2023Survey on Controlable Image Synthesis with Deep Learningcited by 0