mlsanityData to 7 Oct 2026

GRAM: Generative Radiance Manifolds for 3D-Aware Image Generation

arXiv 2112.08867 ↗First posted 16 Dec 2021cs.CVcited by 216

Not on the map. Why?

Cited by 26 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

3D-aware image generative modeling aims to generate 3D-consistent images with explicitly controllable camera poses. Recent works have shown promising results by training neural radiance field (NeRF) generators on unstructured 2D images, but still can not generate highly-realistic images with fine details. A critical reason is that the high memory and computation cost of volumetric representation learning greatly restricts the number of point samples for radiance integration during training. Deficient sampling not only limits the expressive power of the generator to handle fine details but also impedes effective GAN training due to the noise caused by unstable Monte Carlo sampling. We propose a novel approach that regulates point sampling and radiance field learning on 2D manifolds, embodied as a set of learned implicit surfaces in the 3D volume. For each viewing ray, we calculate ray-surface intersections and accumulate their radiance generated by the network. By training and rendering such radiance manifolds, our generator can produce high quality images with realistic fine details and strong visual 3D consistency.

Key facts. Cited by 216 papers in the map (data through 7 Oct 2026). Builds on 56 of its 76 references that are in the map. First posted on arXiv 16 Dec 2021.

Builds on

56 of 76 references.

References · 56

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  4. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  5. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  6. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  7. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  8. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  9. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  10. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  11. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  12. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,295
38 more
  1. 2018Demystifying MMD GANscited by 1,268
  2. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,230
  3. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,176
  4. 2020Neural Sparse Voxel Fieldscited by 959
  5. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  6. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  7. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  8. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  9. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  10. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  11. 2018Which Training Methods for GANs do actually Converge?cited by 672
  12. 2021UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstructioncited by 643
  13. 2020Contrastive Learning for Unpaired Image-to-Image Translationcited by 637
  14. 2020Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humanscited by 634
  15. 2017How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)cited by 524
  16. 2019Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Setcited by 481
  17. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  18. 2020Modular Primitives for High-Performance Differentiable Renderingcited by 439
  19. 2018DeepVoxels: Learning Persistent 3D Feature Embeddingscited by 401
  20. 2019SAL: Sign Agnostic Learning of Shapes from Raw Datacited by 389
  21. 2020StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flowscited by 348
  22. 2020Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstructioncited by 321
  23. 2016Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networkscited by 307
  24. 2018Deep Video Portraitscited by 265
  25. 2020Single-View View Synthesis with Multiplane Imagescited by 263
  26. 2020Disentangled and Controllable Face Image Generation via 3D Imitative-Contrastive Learningcited by 219
  27. 2018Stereo Magnification: Learning View Synthesis using Multiplane Imagescited by 206
  28. 2020Semantic Photo Manipulation with a Generative Image Priorcited by 178
  29. 2018Escaping Plato's Cave: 3D Shape From Adversarial Renderingcited by 170
  30. 2020BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Imagescited by 166
  31. 2019Single Image Portrait Relightingcited by 152
  32. 2021CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesiscited by 149
  33. 2021Unconstrained Scene Generation with Locally Conditioned Radiance Fieldscited by 130
  34. 2018Visual Object Networks: Image Generation with Disentangled 3D Representationcited by 124
  35. 2015Deep Convolutional Inverse Graphics Networkcited by 121
  36. 2019Towards Unsupervised Learning of Generative Models for 3D Controllable Image Synthesiscited by 104
  37. 2021GANcraft: Unsupervised 3D Neural Rendering of Minecraft Worldscited by 98
  38. 2020Lifting 2D StyleGAN for 3D-Aware Face Generationcited by 64

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 49; 2023: 83; 2024: 55; 2025: 17; 2026: 11.

216 papers.

Sort:

By area:

  1. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  2. 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 742
  3. 2023HexPlane: A Fast Representation for Dynamic Scenescited by 582
  4. 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusioncited by 309
  5. 2021StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generationcited by 289
  6. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Modelscited by 273
  7. 2023ImageDream: Image-Prompt Multi-view Diffusion for 3D Generationcited by 256
  8. 2022Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Modelscited by 151
  9. 2022VoxGRAF: Fast 3D-Aware Image Synthesis with Sparse Voxel Gridscited by 133
  10. 2023Locally Attentional SDF Diffusion for Controllable 3D Shape Generationcited by 129
  11. 2022Reconstructing Personalized Semantic Facial NeRF Models From Monocular Videocited by 128
  12. 2022Next3D: Generative Neural Texture Rasterization for 3D-Aware Head Avatarscited by 124
38 more
  1. 2022EVA3D: Compositional 3D Human Generation from 2D Image Collectionscited by 111
  2. 2022EpiGRAF: Rethinking training of 3D GANscited by 108
  3. 2022Generative Neural Articulated Radiance Fieldscited by 101
  4. 2024Arc2Face: A Foundation Model for ID-Consistent Human Facescited by 101
  5. 2024Era3D: High-Resolution Multiview Diffusion using Efficient Row-wise Attentioncited by 99
  6. 2023PanoHead: Geometry-Aware 3D Full-Head Synthesis in 360circcited by 93
  7. 2022DigiFace-1M: 1 Million Digital Face Images for Face Recognitioncited by 92
  8. 2022BAD-NeRF: Bundle Adjusted Deblur Neural Radiance Fieldscited by 80
  9. 2023Neural Residual Radiance Fields for Streamably Free-Viewpoint Videoscited by 77
  10. 2022Object-Compositional Neural Implicit Surfacescited by 74
  11. 2022GRAM-HD: 3D-Consistent Image Generation at High Resolution with Generative Radiance Manifoldscited by 73
  12. 2023SceneDreamer: Unbounded 3D Scene Generation from 2D Image Collectionscited by 72
  13. 20233D-aware Image Generation using 2D Diffusion Modelscited by 69
  14. 2022High-fidelity 3D GAN Inversion by Pseudo-multi-view Optimizationcited by 65
  15. 2023Real-Time Radiance Fields for Single-Image Portrait View Synthesiscited by 64
  16. 2023Texture Generation on 3D Meshes with Point-UV Diffusioncited by 63
  17. 2024SPAD : Spatially Aware Multiview Diffuserscited by 62
  18. 2022DisCoScene: Spatially Disentangled Generative Radiance Fields for Controllable 3D-aware Scene Synthesiscited by 61
  19. 2022AvatarGen: A 3D Generative Model for Animatable Human Avatarscited by 59
  20. 2023CC3D: Layout-Conditioned Generation of Compositional 3D Scenescited by 58
  21. 2022Generative Multiplane Images: Making a 2D GAN 3D-Awarecited by 57
  22. 2023Portrait4D: Learning One-Shot 4D Head Avatar Synthesis using Synthetic Datacited by 55
  23. 2023GeCoNeRF: Few-shot Neural Radiance Fields via Geometric Consistencycited by 54
  24. 20223D GAN Inversion for Controllable Portrait Image Animationcited by 53
  25. 20233D generation on ImageNetcited by 47
  26. 2022AniFaceGAN: Animatable 3D-Aware Face Image Generation for Video Avatarscited by 46
  27. 2024VideoMV: Consistent Multi-View Generation Based on Large Video Generative Modelcited by 45
  28. 20223D GAN Inversion with Pose Optimizationcited by 44
  29. 2023Persistent Nature: A Generative Model of Unbounded 3D Worldscited by 44
  30. 2024GaussianCube: A Structured and Explicit Radiance Representation for 3D Generative Modelingcited by 44
  31. 2024HeadGAP: Few-Shot 3D Head Avatar via Generalizable Gaussian Priorscited by 44
  32. 2023Text2Immersion: Generative Immersive Scene with 3D Gaussianscited by 43
  33. 2023ObjectSDF++: Improved Object-Compositional Neural Implicit Surfacescited by 38
  34. 2022VoLux-GAN: A Generative Model for 3D Face Synthesis with HDRI Relightingcited by 37
  35. 2022Controllable 3D Face Synthesis with Conditional Generative Occupancy Fieldscited by 37
  36. 2022Hand Avatar: Free-Pose Hand Animation and Rendering from Monocular Videocited by 37
  37. 2023Preface: A Data-driven Volumetric Prior for Few-shot Ultra High-resolution Face Synthesiscited by 37
  38. 2024RodinHD: High-Fidelity 3D Avatar Generation with Diffusion Modelscited by 36

50 most cited of 216 All 216 (JSON)