# Single-view Neural Radiance Fields with Depth Teacher

arXiv: https://arxiv.org/abs/2303.09952 · HTML: https://mlsanity.com/paper/2303.09952/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 34 references that are in the map.
- First posted on arXiv 17 Mar 2023.

## Builds on (most cited first, 28 of 28)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [Resolution-robust Large Mask Inpainting with Fourier Convolutions](https://mlsanity.com/paper/2109.07161/) (2021)
- [pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2012.00926/) (2020)
- [SynSin: End-to-end View Synthesis from a Single Image](https://mlsanity.com/paper/1912.08804/) (2019)
- [Single-View View Synthesis with Multiplane Images](https://mlsanity.com/paper/2004.11364/) (2020)
- [Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Image](https://mlsanity.com/paper/2012.09855/) (2020)
- [3D Photography using Context-aware Layered Depth Inpainting](https://mlsanity.com/paper/2004.04727/) (2020)
- [GRF: Learning a General Radiance Field for 3D Representation and Rendering](https://mlsanity.com/paper/2010.04595/) (2020)
- [CodeNeRF: Disentangled Neural Radiance Fields for Object Categories](https://mlsanity.com/paper/2109.01750/) (2021)
- [SinNeRF: Training Neural Radiance Fields on Complex Scenes from a Single Image](https://mlsanity.com/paper/2204.00928/) (2022)
- [MINE: Towards Continuous Depth MPI with NeRF for Novel View Synthesis](https://mlsanity.com/paper/2103.14910/) (2021)
- [3D Ken Burns Effect from a Single Image](https://mlsanity.com/paper/1909.05483/) (2019)
- [Geometry-Free View Synthesis: Transformers and no 3D Priors](https://mlsanity.com/paper/2104.07652/) (2021)
- [ShaRF: Shape-conditioned Radiance Fields from a Single View](https://mlsanity.com/paper/2102.08860/) (2021)
- [PixelSynth: Generating a 3D-Consistent Experience from a Single Image](https://mlsanity.com/paper/2108.05892/) (2021)
- [Worldsheet: Wrapping the World in a 3D Sheet for View Synthesis from a Single Image](https://mlsanity.com/paper/2012.09854/) (2020)
- [Pix2NeRF: Unsupervised Conditional π-GAN for Single Image to Neural Radiance Fields Translation](https://mlsanity.com/paper/2202.13162/) (2022)
- [Learning to Synthesize a 4D RGBD Light Field from a Single Image](https://mlsanity.com/paper/1708.03292/) (2017)
- [Single-View View Synthesis in the Wild with Learned Adaptive Multiplane Images](https://mlsanity.com/paper/2205.11733/) (2022)
- [Look Outside the Room: Synthesizing A Consistent Long-Term 3D Scene Video from A Single Image](https://mlsanity.com/paper/2203.09457/) (2022)
- [Learning Stereo from Single Images](https://mlsanity.com/paper/2008.01484/) (2020)
- [AUTO3D: Novel view synthesis through unsupervisely learned variational viewpoint and global 3D representation](https://mlsanity.com/paper/2007.06620/) (2020)
- [Embedding Novel Views in a Single JPEG Image](https://mlsanity.com/paper/2108.13003/) (2021)
- [Remote Sensing Novel View Synthesis with Implicit Multiplane Representations](https://mlsanity.com/paper/2205.08908/) (2022)
- [Unsupervised Novel View Synthesis from a Single Image](https://mlsanity.com/paper/2102.03285/) (2021)
- [PVSeRF: Joint Pixel-, Voxel- and Surface-Aligned Radiance Field for Single-Image Novel View Synthesis](https://mlsanity.com/paper/2202.04879/) (2022)

## Built on by (most cited first, 1 of 1)

- [Semantically-aware Neural Radiance Fields for Visual Scene Understanding: A Comprehensive Review](https://mlsanity.com/paper/2402.11141/) (2024)

## Abstract (arXiv)

Neural Radiance Fields (NeRF) have been proposed for photorealistic novel view rendering. However, it requires many different views of one scene for training. Moreover, it has poor generalizations to new scenes and requires retraining or fine-tuning on each scene. In this paper, we develop a new NeRF model for novel view synthesis using only a single image as input. We propose to combine the (coarse) planar rendering and the (fine) volume rendering to achieve higher rendering quality and better generalizations. We also design a depth teacher net that predicts dense pseudo depth maps to supervise the joint rendering mechanism and boost the learning of consistent 3D geometry. We evaluate our method on three challenging datasets. It outperforms state-of-the-art single-view NeRFs by achieving 5~20% improvements in PSNR and reducing 20~50% of the errors in the depth rendering. It also shows excellent generalization abilities to unseen data without the need to fine-tune on each new scene.