Single-view Neural Radiance Fields with Depth Teacher
Not on the map. Why?
Abstract (arXiv)
Neural Radiance Fields (NeRF) have been proposed for photorealistic novel view rendering. However, it requires many different views of one scene for training. Moreover, it has poor generalizations to new scenes and requires retraining or fine-tuning on each scene. In this paper, we develop a new NeRF model for novel view synthesis using only a single image as input. We propose to combine the (coarse) planar rendering and the (fine) volume rendering to achieve higher rendering quality and better generalizations. We also design a depth teacher net that predicts dense pseudo depth maps to supervise the joint rendering mechanism and boost the learning of consistent 3D geometry. We evaluate our method on three challenging datasets. It outperforms state-of-the-art single-view NeRFs by achieving 5~20% improvements in PSNR and reducing 20~50% of the errors in the depth rendering. It also shows excellent generalization abilities to unseen data without the need to fine-tune on each new scene.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 28 of its 34 references that are in the map. First posted on arXiv 17 Mar 2023.
Builds on
28 of 34 references.
References · 28
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2021Vision Transformers for Dense Predictioncited by 1,789
- 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
- 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 730
- 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
- 2019SynSin: End-to-end View Synthesis from a Single Imagecited by 364
- 2020Single-View View Synthesis with Multiplane Imagescited by 263
- 2020Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Imagecited by 239
- 20203D Photography using Context-aware Layered Depth Inpaintingcited by 206
- 2020GRF: Learning a General Radiance Field for 3D Representation and Renderingcited by 183
- 2021CodeNeRF: Disentangled Neural Radiance Fields for Object Categoriescited by 183
- 2022SinNeRF: Training Neural Radiance Fields on Complex Scenes from a Single Imagecited by 165
16 more
- 2021MINE: Towards Continuous Depth MPI with NeRF for Novel View Synthesiscited by 133
- 20193D Ken Burns Effect from a Single Imagecited by 128
- 2021Geometry-Free View Synthesis: Transformers and no 3D Priorscited by 99
- 2021ShaRF: Shape-conditioned Radiance Fields from a Single Viewcited by 92
- 2021PixelSynth: Generating a 3D-Consistent Experience from a Single Imagecited by 82
- 2020Worldsheet: Wrapping the World in a 3D Sheet for View Synthesis from a Single Imagecited by 77
- 2022Pix2NeRF: Unsupervised Conditional π-GAN for Single Image to Neural Radiance Fields Translationcited by 73
- 2017Learning to Synthesize a 4D RGBD Light Field from a Single Imagecited by 60
- 2022Single-View View Synthesis in the Wild with Learned Adaptive Multiplane Imagescited by 59
- 2022Look Outside the Room: Synthesizing A Consistent Long-Term 3D Scene Video from A Single Imagecited by 55
- 2020Learning Stereo from Single Imagescited by 44
- 2020AUTO3D: Novel view synthesis through unsupervisely learned variational viewpoint and global 3D representationcited by 14
- 2021Embedding Novel Views in a Single JPEG Imagecited by 10
- 2022Remote Sensing Novel View Synthesis with Implicit Multiplane Representationscited by 9
- 2021Unsupervised Novel View Synthesis from a Single Imagecited by 8
- 2022PVSeRF: Joint Pixel-, Voxel- and Surface-Aligned Radiance Field for Single-Image Novel View Synthesiscited by 5
Built on by
Citing papers per year: 2024: 1.
1 paper.