mlsanityData to 14 Oct 2026

CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesis

arXiv 2110.09788 ↗First posted 19 Oct 2021cs.CV eess.IVcited by 149

Not on the map. Why?

Cited by 20 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The style-based GAN (StyleGAN) architecture achieved state-of-the-art results for generating high-quality images, but it lacks explicit and precise control over camera poses. The recently proposed NeRF-based GANs made great progress towards 3D-aware generators, but they are unable to generate high-quality images yet. This paper presents CIPS-3D, a style-based, 3D-aware generator that is composed of a shallow NeRF network and a deep implicit neural representation (INR) network. The generator synthesizes each pixel value independently without any spatial convolution or upsampling operation. In addition, we diagnose the problem of mirror symmetry that implies a suboptimal solution and solve it by introducing an auxiliary discriminator. Trained on raw, single-view images, CIPS-3D sets new records for 3D-aware image synthesis with an impressive FID of 6.97 for images at the 256×256 resolution on FFHQ. We also demonstrate several interesting directions for CIPS-3D such as transfer learning and 3D-aware face stylization. The synthesis results are best viewed as videos, so we recommend the readers to check our github project at https://github.com/PeterouZh/CIPS-3D

Key facts. Cited by 149 papers in the map (data through 14 Oct 2026). Builds on 48 of its 62 references that are in the map. First posted on arXiv 19 Oct 2021.

Builds on

48 of 62 references.

References · 48

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  3. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  4. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  5. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  6. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  7. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  8. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,238
  9. 2017Improved Training of Wasserstein GANscited by 2,235
  10. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,054
  11. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  12. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,964
36 more
  1. 2018Demystifying MMD GANscited by 1,293
  2. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,232
  3. 2020Training Generative Adversarial Networks with Limited Datacited by 1,186
  4. 2021Alias-Free Generative Adversarial Networkscited by 1,184
  5. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,098
  6. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  7. 2020NeRF++: Analyzing and Improving Neural Radiance Fieldscited by 861
  8. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  9. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 785
  10. 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 740
  11. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  12. 2019Interpreting the Latent Space of GANs for Semantic Face Editingcited by 695
  13. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 689
  14. 2018Which Training Methods for GANs do actually Converge?cited by 679
  15. 2020GANSpace: Discovering Interpretable GAN Controlscited by 614
  16. 2020Learning Continuous Image Representation with Local Implicit Image Functioncited by 552
  17. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  18. 2020Closed-Form Factorization of Latent Semantics in GANscited by 408
  19. 2019On the "steerability" of generative adversarial networkscited by 248
  20. 2019Learning Shape Templates with Structured Implicit Functionscited by 242
  21. 2021Learning Object-Compositional Neural Radiance Field for Editable Scene Renderingcited by 228
  22. 2021CodeNeRF: Disentangled Neural Radiance Fields for Object Categoriescited by 183
  23. 2019Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wildcited by 180
  24. 2018Escaping Plato's Cave: 3D Shape From Adversarial Renderingcited by 170
  25. 2020Adversarial Generation of Continuous Imagescited by 135
  26. 2021Unconstrained Scene Generation with Locally Conditioned Radiance Fieldscited by 130
  27. 2018Visual Object Networks: Image Generation with Disentangled 3D Representationcited by 124
  28. 2019Semantic Hierarchy Emerges in Deep Generative Representations for Scene Synthesiscited by 106
  29. 2020Image Generators with Conditionally-Independent Pixel Synthesiscited by 106
  30. 2020Resolution Dependent GAN Interpolation for Controllable Image Synthesis Between Domainscited by 96
  31. 2020Do 2D GANs Know 3D Shape? Unsupervised 3D shape reconstruction from 2D Image GANscited by 63
  32. 2021UltraSR: Spatial Encoding is a Missing Key for Implicit Image Function-based Arbitrary-Scale Super-Resolutioncited by 57
  33. 2021CAMPARI: Camera-Aware Decomposed Generative Neural Radiance Fieldscited by 55
  34. 2019Deep Meta Functionals for Shape Representationcited by 53
  35. 2021On Aliased Resizing and Surprising Subtleties in GAN Evaluationcited by 51
  36. 2020Unsupervised Image-to-Image Translation via Pre-trained StyleGAN2 Networkcited by 36

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 12; 2022: 55; 2023: 55; 2024: 21; 2025: 5; 2026: 1.

149 papers.

Sort:

By area:

  1. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  2. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 699
  3. 2021RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputscited by 615
  4. 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 553
  5. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  6. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  7. 2021Advances in Neural Renderingcited by 365
  8. 2023OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generationcited by 326
  9. 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusioncited by 309
  10. 2021StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generationcited by 289
  11. 20223D Neural Field Generation using Triplane Diffusioncited by 274
  12. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Modelscited by 274
38 more
  1. 2021HeadNeRF: A Real-time NeRF-based Parametric Head Modelcited by 228
  2. 2021GRAM: Generative Radiance Manifolds for 3D-Aware Image Generationcited by 216
  3. 2022DiffRF: Rendering-Guided 3D Radiance Field Diffusioncited by 204
  4. 2023Flexible Isosurface Extraction for Gradient-Based Mesh Optimizationcited by 139
  5. 2022GAUDI: A Neural Architect for Immersive 3D Scene Generationcited by 136
  6. 2022VoxGRAF: Fast 3D-Aware Image Synthesis with Sparse Voxel Gridscited by 133
  7. 2022Reconstructing Personalized Semantic Facial NeRF Models From Monocular Videocited by 128
  8. 2022Next3D: Generative Neural Texture Rasterization for 3D-Aware Head Avatarscited by 124
  9. 2021FENeRF: Face Editing in Neural Radiance Fieldscited by 112
  10. 2022EpiGRAF: Rethinking training of 3D GANscited by 108
  11. 2022Generative Neural Articulated Radiance Fieldscited by 101
  12. 2023Compositional 3D Scene Generation using Locally Conditioned Diffusioncited by 101
  13. 2022Pix2NeRF: Unsupervised Conditional π-GAN for Single Image to Neural Radiance Fields Translationcited by 73
  14. 2022GRAM-HD: 3D-Consistent Image Generation at High Resolution with Generative Radiance Manifoldscited by 73
  15. 2022Unified Implicit Neural Stylizationcited by 67
  16. 2022Texturify: Generating Textures on 3D Shape Surfacescited by 65
  17. 2022High-fidelity 3D GAN Inversion by Pseudo-multi-view Optimizationcited by 65
  18. 2023Real-Time Radiance Fields for Single-Image Portrait View Synthesiscited by 64
  19. 2021SemanticStyleGAN: Learning Compositional Generative Priors for Controllable Image Synthesis and Editingcited by 58
  20. 2023CC3D: Layout-Conditioned Generation of Compositional 3D Scenescited by 58
  21. 2022Generative Multiplane Images: Making a 2D GAN 3D-Awarecited by 57
  22. 20223D GAN Inversion for Controllable Portrait Image Animationcited by 53
  23. 2023Large-Vocabulary 3D Diffusion Model with Transformercited by 48
  24. 20233D generation on ImageNetcited by 47
  25. 2022Signal Processing for Implicit Neural Representationscited by 46
  26. 20223D GAN Inversion with Pose Optimizationcited by 44
  27. 20223D GAN Inversion with Facial Symmetry Priorcited by 39
  28. 2022StudioGAN: A Taxonomy and Benchmark of GANs for Image Synthesiscited by 38
  29. 2022VoLux-GAN: A Generative Model for 3D Face Synthesis with HDRI Relightingcited by 37
  30. 2023Preface: A Data-driven Volumetric Prior for Few-shot Ultra High-resolution Face Synthesiscited by 37
  31. 2023PartNeRF: Generating Part-Aware Editable 3D Shapes without 3D Supervisioncited by 36
  32. 2022State-of-the-Art in the Architecture, Methods and Applications of StyleGANcited by 34
  33. 2023OmniAvatar: Geometry-Guided Controllable 3D Head Synthesiscited by 34
  34. 2021HVTR: Hybrid Volumetric-Textural Rendering for Human Avatarscited by 33
  35. 2022NeRFFaceEditing: Disentangled Face Editing in Neural Radiance Fieldscited by 33
  36. 2022HumanGen: Generating Human Radiance Fields with Explicit Priorscited by 33
  37. 2022DATID-3D: Diversity-Preserved Domain Adaptation Using Text-to-Image Diffusion for 3D Generative Modelcited by 32
  38. 2024Holo-Relighting: Controllable Volumetric Portrait Relighting from a Single Imagecited by 32

50 most cited of 149 All 149 (JSON)