# NeRFVS: Neural Radiance Fields for Free View Synthesis via Geometry Scaffolds

arXiv: https://arxiv.org/abs/2304.06287 · HTML: https://mlsanity.com/paper/2304.06287/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 38 references that are in the map.
- First posted on arXiv 13 Apr 2023.

## Builds on (most cited first, 23 of 23)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [Depth-supervised NeRF: Fewer Views and Faster Training for Free](https://mlsanity.com/paper/2107.02791/) (2021)
- [IBRNet: Learning Multi-View Image-Based Rendering](https://mlsanity.com/paper/2102.13090/) (2021)
- [RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputs](https://mlsanity.com/paper/2112.00724/) (2021)
- [NICE-SLAM: Neural Implicit Scalable Encoding for SLAM](https://mlsanity.com/paper/2112.12130/) (2021)
- [iMAP: Implicit Mapping and Positioning in Real-Time](https://mlsanity.com/paper/2103.12352/) (2021)
- [MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstruction](https://mlsanity.com/paper/2206.00665/) (2022)
- [Dense Depth Priors for Neural Radiance Fields from Sparse Input Views](https://mlsanity.com/paper/2112.03288/) (2021)
- [NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video](https://mlsanity.com/paper/2104.00681/) (2021)
- [Free View Synthesis](https://mlsanity.com/paper/2008.05511/) (2020)
- [NerfingMVS: Guided Optimization of Neural Radiance Fields for Indoor Multi-view Stereo](https://mlsanity.com/paper/2109.01129/) (2021)
- [InfoNeRF: Ray Entropy Minimization for Few-Shot Neural Volume Rendering](https://mlsanity.com/paper/2112.15399/) (2021)
- [Stable View Synthesis](https://mlsanity.com/paper/2011.07233/) (2020)
- [SinNeRF: Training Neural Radiance Fields on Complex Scenes from a Single Image](https://mlsanity.com/paper/2204.00928/) (2022)
- [NeuRIS: Neural Reconstruction of Indoor Scenes Using Normal Priors](https://mlsanity.com/paper/2206.13597/) (2022)
- [Neural 3D Scene Reconstruction with the Manhattan-world Assumption](https://mlsanity.com/paper/2205.02836/) (2022)
- [Free-viewpoint Indoor Neural Relighting from Multi-view Stereo](https://mlsanity.com/paper/2106.13299/) (2021)
- [Ray Priors through Reprojection: Improving Neural Radiance Fields for Novel View Extrapolation](https://mlsanity.com/paper/2205.05922/) (2022)
- [PhotoScene: Photorealistic Material and Lighting Transfer for Indoor Scenes](https://mlsanity.com/paper/2207.00757/) (2022)

## Built on by (most cited first, 16 of 16)

- [Depth-Regularized Optimization for 3D Gaussian Splatting in Few-Shot Images](https://mlsanity.com/paper/2311.13398/) (2023)
- [VEGS: View Extrapolation of Urban Scenes in 3D Gaussian Splatting using Learned Priors](https://mlsanity.com/paper/2407.02945/) (2024)
- [Efficient Deformable Tissue Reconstruction via Orthogonal Neural Plane](https://mlsanity.com/paper/2312.15253/) (2023)
- [FewViewGS: Gaussian Splatting with Few View Matching and Multi-stage Training](https://mlsanity.com/paper/2411.02229/) (2024)
- [LidaRF: Delving into Lidar for Neural Radiance Field on Street Scenes](https://mlsanity.com/paper/2405.00900/) (2024)
- [DenseSplat: Densifying Gaussian Splatting SLAM with Neural Radiance Prior](https://mlsanity.com/paper/2502.09111/) (2025)
- [Extrapolated Urban View Synthesis Benchmark](https://mlsanity.com/paper/2412.05256/) (2024)
- [Neural Radiance Field-based Visual Rendering: A Comprehensive Review](https://mlsanity.com/paper/2404.00714/) (2024)
- [The NeRF Signature: Codebook-Aided Watermarking for Neural Radiance Fields](https://mlsanity.com/paper/2502.19125/) (2025)
- [CoMapGS: Covisibility Map-based Gaussian Splatting for Sparse Novel View Synthesis](https://mlsanity.com/paper/2503.20998/) (2025)
- [LighthouseGS: Indoor Structure-aware 3D Gaussian Splatting for Panorama-Style Mobile Captures](https://mlsanity.com/paper/2507.06109/) (2025)
- [VoxNeRF: Bridging Voxel Representation and Neural Radiance Fields for Enhanced Indoor View Synthesis](https://mlsanity.com/paper/2311.05289/) (2023)
- [Methods and strategies for improving the novel view synthesis quality of neural radiation field](https://mlsanity.com/paper/2401.12451/) (2024)
- [UrbanCraft: Urban View Extrapolation via Hierarchical Sem-Geometric Priors](https://mlsanity.com/paper/2505.23434/) (2025)
- [Geometry-Aware Scene Configurations for Novel View Synthesis](https://mlsanity.com/paper/2510.09880/) (2025)
- [CLEAR-NeRF: Collinearity and Local-region Enhanced Accurate 3D Reconstruction in Unbounded Scenes](https://mlsanity.com/paper/2605.28125/) (2026)

## Abstract (arXiv)

We present NeRFVS, a novel neural radiance fields (NeRF) based method to enable free navigation in a room. NeRF achieves impressive performance in rendering images for novel views similar to the input views while suffering for novel views that are significantly different from the training views. To address this issue, we utilize the holistic priors, including pseudo depth maps and view coverage information, from neural reconstruction to guide the learning of implicit neural representations of 3D indoor scenes. Concretely, an off-the-shelf neural reconstruction method is leveraged to generate a geometry scaffold. Then, two loss functions based on the holistic priors are proposed to improve the learning of NeRF: 1) A robust depth loss that can tolerate the error of the pseudo depth map to guide the geometry learning of NeRF; 2) A variance loss to regularize the variance of implicit neural representations to reduce the geometry and color ambiguity in the learning procedure. These two loss functions are modulated during NeRF optimization according to the view coverage information to reduce the negative influence brought by the view coverage imbalance. Extensive results demonstrate that our NeRFVS outperforms state-of-the-art view synthesis methods quantitatively and qualitatively on indoor scenes, achieving high-fidelity free navigation results.