mlsanityData to 7 Oct 2026

3D-aware Blending with Generative NeRFs

arXiv 2302.06608 ↗First posted 13 Feb 2023cs.CV cs.GR cs.LGcited by 13

Not on the map. Why?

Abstract (arXiv)

Image blending aims to combine multiple images seamlessly. It remains challenging for existing 2D-based methods, especially when input images are misaligned due to differences in 3D camera poses and object shapes. To tackle these issues, we propose a 3D-aware blending method using generative Neural Radiance Fields (NeRF), including two key components: 3D-aware alignment and 3D-aware blending. For 3D-aware alignment, we first estimate the camera pose of the reference image with respect to generative NeRFs and then perform 3D local alignment for each part. To further leverage 3D information of the generative NeRF, we propose 3D-aware blending that directly blends images on the NeRF's latent representation space, rather than raw pixel space. Collectively, our method outperforms existing 2D baselines, as validated by extensive quantitative and qualitative evaluations with FFHQ and AFHQ-Cat.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 75 of its 118 references that are in the map. First posted on arXiv 13 Feb 2023.

Builds on

75 of 118 references.

References · 75

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  6. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  7. 2022Generative Adversarial Networkscited by 5,603
  8. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  9. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  10. 2016Improved Techniques for Training GANscited by 2,938
  11. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  12. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
38 more
  1. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  2. 2018Demystifying MMD GANscited by 1,268
  3. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  4. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  5. 2016Colorful Image Colorizationcited by 1,113
  6. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,095
  7. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  8. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  9. 2017VoxCeleb: a large-scale speaker identification datasetcited by 714
  10. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  11. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  12. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  13. 2018BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentationcited by 645
  14. 2021Label-Efficient Semantic Segmentation with Diffusion Modelscited by 495
  15. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  16. 2021Pivotal Tuning for Latent-based Editing of Real Imagescited by 427
  17. 2020In-Domain GAN Inversion for Real Image Editingcited by 420
  18. 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 370
  19. 2021CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fieldscited by 369
  20. 2016Generative Visual Manipulation on the Natural Image Manifoldcited by 348
  21. 2019Deep Closest Point: Learning Representations for Point Cloud Registrationcited by 318
  22. 2021StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generationcited by 289
  23. 2021DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effortcited by 264
  24. 2020Deep Global Registrationcited by 228
  25. 2021Editing Conditional Radiance Fieldscited by 222
  26. 2021GRAM: Generative Radiance Manifolds for 3D-Aware Image Generationcited by 216
  27. 2022NeRF-Editing: Geometry Editing of Neural Radiance Fieldscited by 216
  28. 2020Editing in Style: Uncovering the Local Semantics of GANscited by 183
  29. 2020Semantic Photo Manipulation with a Generative Image Priorcited by 178
  30. 2018Escaping Plato's Cave: 3D Shape From Adversarial Renderingcited by 170
  31. 2021CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesiscited by 149
  32. 2018Visual Object Networks: Image Generation with Disentangled 3D Representationcited by 124
  33. 2019Deep Learning for Deepfakes Creation and Detection: A Surveycited by 102
  34. 2021Exploiting Spatial Dimensions of Latent in GAN for Real-time Image Editingcited by 91
  35. 2018ST-GAN: Spatial Transformer Generative Adversarial Networks for Image Compositingcited by 90
  36. 2020Improved StyleGAN Embedding: Where are the Good Latents?cited by 88
  37. 2021HyperInverter: Improving StyleGAN Inversion via Hypernetworkcited by 85
  38. 2020Transforming and Projecting Images into Class-conditional Generative Networkscited by 76

50 most cited of 75 All 75 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 5; 2025: 1; 2026: 2.

13 papers.

Sort:

By area:

  1. 2023CAD: Photorealistic 3D Generation via Adversarial Distillationcited by 18
  2. 2023LatentEditor: Text Driven Local Editing of 3D Scenescited by 17
  3. 2023Free-Editor: Zero-shot Text-driven 3D Scene Editingcited by 14
  4. 2024Localized Gaussian Splatting Editing with Contextual Awarenesscited by 13
  5. 2025Compass Control: Multi Object Orientation Control for Text-to-Image Generationcited by 11
  6. 2023Language-driven Object Fusion into Neural Radiance Fields with Pose-Conditioned Dataset Updatescited by 9
  7. 2024PEGASUS: Personalized Generative 3D Avatars with Composable Attributescited by 7
  8. 2024Noise Calibration: Plug-and-play Content-Preserving Video Enhancement using Pre-trained Video Diffusion Modelscited by 5
  9. 2023TEGLO: High Fidelity Canonical Texture Mapping from Single-View Imagescited by 4
  10. 2026SeeThrough3D: Occlusion Aware 3D Control in Text-to-Image Generationcited by 4
  11. 2026The Devil Is in the Leakage: A Disentangled Dual-Purification Framework for High-Fidelity Hairstyle Transfercited by 1
  12. 2024DepGAN: Leveraging Depth Maps for Handling Occlusions and Transparency in Image Compositioncited by 0
1 more
  1. 2024ArCSEM: Artistic Colorization of SEM Images via Gaussian Splattingcited by 0