mlsanityData to 7 Oct 2026

3D-aware Conditional Image Synthesis

arXiv 2302.08509 ↗First posted 16 Feb 2023cs.CV cs.GR cs.LGcited by 24

Not on the map. Why?

Abstract (arXiv)

We propose pix2pix3D, a 3D-aware conditional generative model for controllable photorealistic image synthesis. Given a 2D label map, such as a segmentation or edge map, our model learns to synthesize a corresponding image from different viewpoints. To enable explicit 3D user control, we extend conditional generative models with neural radiance fields. Given widely-available monocular images and label map pairs, our model learns to assign a label to every 3D point in addition to color and density, which enables it to render the image and pixel-aligned label map simultaneously. Finally, we build an interactive system that allows users to edit the label map from any viewpoint and generate outputs accordingly.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 75 of its 90 references that are in the map. First posted on arXiv 16 Feb 2023.

Builds on

75 of 90 references.

References · 75

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  4. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  5. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  6. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  7. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  8. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  9. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  10. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  11. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  12. 2014Conditional Generative Adversarial Netscited by 2,152
38 more
  1. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,607
  2. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  3. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,295
  4. 2022TensoRF: Tensorial Radiance Fieldscited by 1,275
  5. 2018Demystifying MMD GANscited by 1,268
  6. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,230
  7. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  8. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  9. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,176
  10. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,095
  11. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  12. 2020NeRF++: Analyzing and Improving Neural Radiance Fieldscited by 857
  13. 2019CNN-generated images are surprisingly easy to spot... for nowcited by 842
  14. 2018Multimodal Unsupervised Image-to-Image Translationcited by 795
  15. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  16. 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 735
  17. 2021Depth-supervised NeRF: Fewer Views and Faster Training for Freecited by 732
  18. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  19. 2017Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networkscited by 707
  20. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  21. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  22. 2018Which Training Methods for GANs do actually Converge?cited by 672
  23. 2021BARF: Bundle-Adjusting Neural Radiance Fieldscited by 641
  24. 2020Contrastive Learning for Unpaired Image-to-Image Translationcited by 637
  25. 2019MaskGAN: Towards Diverse and Interactive Facial Image Manipulationcited by 624
  26. 2021RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputscited by 614
  27. 2017Unsupervised Image-to-Image Translation Networkscited by 602
  28. 2021NICE-SLAM: Neural Implicit Scalable Encoding for SLAMcited by 586
  29. 2021Designing an Encoder for StyleGAN Image Manipulationcited by 542
  30. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  31. 2021iMAP: Implicit Mapping and Positioning in Real-Timecited by 476
  32. 2021Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesiscited by 464
  33. 2020In-Domain GAN Inversion for Real Image Editingcited by 420
  34. 2020Closed-Form Factorization of Latent Semantics in GANscited by 407
  35. 2016Generative Visual Manipulation on the Natural Image Manifoldcited by 348
  36. 2021Deep ViT Features as Dense Visual Descriptorscited by 327
  37. 2017Toward Multimodal Image-to-Image Translationcited by 295
  38. 2021StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generationcited by 289

50 most cited of 75 All 75 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 11; 2024: 8; 2025: 4; 2026: 1.

24 papers.

Sort:

By area:

  1. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 638
  2. 2023Tri-MipRF: Tri-Mip Representation for Efficient Anti-Aliasing Neural Radiance Fieldscited by 142
  3. 2023Motion-Conditioned Diffusion Model for Controllable Video Synthesiscited by 99
  4. 2023CC3D: Layout-Conditioned Generation of Compositional 3D Scenescited by 58
  5. 2023Control3Diff: Learning Controllable 3D Diffusion Models from Single-view Imagescited by 27
  6. 2023GINA-3D: Learning to Generate Implicit Neural Assets in the Wildcited by 25
  7. 2024Synthetic Face Datasets Generation via Latent Space Exploration from Brownian Identity Diffusioncited by 12
  8. 2024Diffusion-driven GAN Inversion for Multi-Modal Face Image Generationcited by 12
  9. 2023MinD-3D: Reconstruct High-quality 3D objects in Human Braincited by 11
  10. 2025Seeing through Satellite Images at Street Viewscited by 8
  11. 2023Multi-task View Synthesis with Neural Radiance Fieldscited by 5
  12. 2023Efficient Text-Guided 3D-Aware Portrait Generation with Score Distillation Sampling on Distributioncited by 4
12 more
  1. 2023MaTe3D: Mask-guided Text-based 3D-aware Portrait Editingcited by 3
  2. 2024SemanticHuman-HD: High-Resolution Semantic Disentangled 3D Human Generationcited by 3
  3. 2024Exploring the Potentials and Challenges of Deep Generative Models in Product Design Conceptioncited by 3
  4. 20243D-aware Image Generation and Editing with Multi-modal Conditionscited by 2
  5. 2025SuperNeRF-GAN: A Universal 3D-Consistent Super-Resolution Framework for Efficient and Enhanced 3D-Aware Image Synthesiscited by 2
  6. 2025XOCT: Enhancing OCT to OCTA Translation via Cross-Dimensional Supervised Multi-Scale Feature Learningcited by 2
  7. 2023DatasetNeRF: Efficient 3D-aware Data Factory with Generative Radiance Fieldscited by 1
  8. 2025S3D: Sketch-Driven 3D Model Generationcited by 1
  9. 2024Text-Driven Diverse Facial Texture Generation via Progressive Latent-Space Refinementcited by 0
  10. 2024A Reference-Based 3D Semantic-Aware Framework for Accurate Local Facial Attribute Editingcited by 0
  11. 2024Pomo3D: 3D-Aware Portrait Accessorizing and Morecited by 0
  12. 2026SketchFaceGS: Real-Time Sketch-Driven Face Editing and Generation with Gaussian Splattingcited by 0