mlsanityData to 14 Oct 2026

CompNVS: Novel View Synthesis with Scene Completion

arXiv 2207.11467 ↗First posted 23 Jul 2022cs.CV cs.AIcited by 9

Not on the map. Why?

Abstract (arXiv)

We introduce a scalable framework for novel view synthesis from RGB-D images with largely incomplete scene coverage. While generative neural approaches have demonstrated spectacular results on 2D images, they have not yet achieved similar photorealistic results in combination with scene completion where a spatial 3D scene understanding is essential. To this end, we propose a generative pipeline performing on a sparse grid-based neural scene representation to complete unobserved scene parts via a learned distribution of scenes in a 2.5D-3D-2.5D manner. We process encoded image features in 3D space with a geometry completion network and a subsequent texture inpainting network to extrapolate the missing area. Photorealistic image sequences can be finally obtained via consistency-relevant differentiable rendering. Comprehensive experiments show that the graphical outputs of our method outperform the state of the art, especially within unobserved scene parts.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 27 of its 30 references that are in the map. First posted on arXiv 23 Jul 2022.

Builds on

27 of 30 references.

References · 27

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  4. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  5. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,238
  6. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
  7. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,208
  8. 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
  9. 2020Neural Sparse Voxel Fieldscited by 959
  10. 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 904
  11. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 785
  12. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
15 more
  1. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 689
  2. 2018Free-Form Image Inpainting with Gated Convolutioncited by 594
  3. 2018Generative Image Inpainting with Contextual Attentioncited by 580
  4. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 511
  5. 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 454
  6. 2019SynSin: End-to-end View Synthesis from a Single Imagecited by 364
  7. 2017Toward Multimodal Image-to-Image Translationcited by 296
  8. 2020BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Imagescited by 166
  9. 2021Unconstrained Scene Generation with Locally Conditioned Radiance Fieldscited by 130
  10. 2021PixelSynth: Generating a 3D-Consistent Experience from a Single Imagecited by 82
  11. 2020Generative Sparse Detection Networks for 3D Single-shot Object Detectioncited by 69
  12. 2020World-Consistent Video-to-Video Synthesiscited by 69
  13. 2019HoloGAN: Unsupervised learning of 3D representations from natural imagescited by 59
  14. 2020SPSG: Self-Supervised Photometric Scene Generation from RGB-D Scanscited by 34
  15. 2020Sat2Vid: Street-view Panoramic Video Synthesis from a Single Satellite Imagecited by 25

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 3; 2024: 1; 2025: 2.

9 papers.

  1. 2022Removing Objects From Neural Radiance Fieldscited by 66
  2. 2023Watch Your Steps: Local Image and Scene Editing by Text Instructionscited by 50
  3. 2023Reference-guided Controllable Inpainting of Neural Radiance Fieldscited by 40
  4. 2024Sat2Scene: 3D Urban Scene Generation from Satellite Images with Diffusioncited by 27
  5. 2025WorldGen: From Text to Traversable and Interactive 3D Worldscited by 19
  6. 2022SPARF: Large-Scale Learning of 3D Sparse Radiance Fields from Few Input Imagescited by 17
  7. 2022Free-form 3D Scene Inpainting with Dual-stream GANcited by 6
  8. 2023SparseGNV: Generating Novel Views of Indoor Scenes with Sparse Input Viewscited by 2
  9. 2025Beyond Darkness: Thermal-Supervised 3D Gaussian Splatting for Low-Light Novel View Synthesiscited by 1