# SO-NeRF: Active View Planning for NeRF using Surrogate Objectives

arXiv: https://arxiv.org/abs/2312.03266 · HTML: https://mlsanity.com/paper/2312.03266/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 51 references that are in the map.
- First posted on arXiv 6 Dec 2023.

## Builds on (most cited first, 30 of 38)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [Objaverse: A Universe of Annotated 3D Objects](https://mlsanity.com/paper/2212.08051/) (2022)
- [TensoRF: Tensorial Radiance Fields](https://mlsanity.com/paper/2203.09517/) (2022)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [On the Convergence of Adam and Beyond](https://mlsanity.com/paper/1904.09237/) (2019)
- [MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2103.15595/) (2021)
- [Nerfstudio: A Modular Framework for Neural Radiance Field Development](https://mlsanity.com/paper/2302.04264/) (2023)
- [NICE-SLAM: Neural Implicit Scalable Encoding for SLAM](https://mlsanity.com/paper/2112.12130/) (2021)
- [FastNeRF: High-Fidelity Neural Rendering at 200FPS](https://mlsanity.com/paper/2103.10380/) (2021)
- [Baking Neural Radiance Fields for Real-Time View Synthesis](https://mlsanity.com/paper/2103.14645/) (2021)
- [Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesis](https://mlsanity.com/paper/2104.00677/) (2021)
- [V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Prediction](https://mlsanity.com/paper/2008.07519/) (2020)
- [Generative Novel View Synthesis with 3D-Aware Diffusion Models](https://mlsanity.com/paper/2304.02602/) (2023)
- [Robust Dynamic Radiance Fields](https://mlsanity.com/paper/2301.02239/) (2023)
- [Neural Rays for Occlusion-aware Image-based Rendering](https://mlsanity.com/paper/2107.13421/) (2021)
- [NeRF-SLAM: Real-Time Dense Monocular SLAM with Neural Radiance Fields](https://mlsanity.com/paper/2210.13641/) (2022)
- [ActiveNeRF: Learning where to See with Uncertainty Estimation](https://mlsanity.com/paper/2209.08546/) (2022)
- [DiffusioNeRF: Regularizing Neural Radiance Fields with Denoising Diffusion Models](https://mlsanity.com/paper/2302.12231/) (2023)
- [Bayes' Rays: Uncertainty Quantification for Neural Radiance Fields](https://mlsanity.com/paper/2309.03185/) (2023)
- [VR-NeRF: High-Fidelity Virtualized Walkable Spaces](https://mlsanity.com/paper/2311.02542/) (2023)
- [Stochastic Neural Radiance Fields: Quantifying Uncertainty in Implicit 3D Representations](https://mlsanity.com/paper/2109.02123/) (2021)
- [FoV-NeRF: Foveated Neural Radiance Fields for Virtual Reality](https://mlsanity.com/paper/2103.16365/) (2021)
- [Long-Term Photometric Consistent Novel View Synthesis with Diffusion Models](https://mlsanity.com/paper/2304.10700/) (2023)
- [NeurAR: Neural Uncertainty for Autonomous 3D Reconstruction with Implicit Neural Representations](https://mlsanity.com/paper/2207.10985/) (2022)
- [NeU-NBV: Next Best View Planning Using Uncertainty Estimation in Image-Based Neural Rendering](https://mlsanity.com/paper/2303.01284/) (2023)

## Built on by (most cited first, 5 of 5)

- [GauSS-MI: Gaussian Splatting Shannon Mutual Information for Active 3D Reconstruction](https://mlsanity.com/paper/2504.21067/) (2025)
- [NextBestPath: Efficient 3D Mapping of Unseen Environments](https://mlsanity.com/paper/2502.05378/) (2025)
- [NeRF Analogies: Example-Based Visual Attribute Transfer for NeRFs](https://mlsanity.com/paper/2402.08622/) (2024)
- [Diversity-Driven View Subset Selection for Indoor Novel View Synthesis](https://mlsanity.com/paper/2409.07098/) (2024)
- [VIN-NBV: A View Introspection Network for Next-Best-View Selection](https://mlsanity.com/paper/2505.06219/) (2025)

## Abstract (arXiv)

Despite the great success of Neural Radiance Fields (NeRF), its data-gathering process remains vague with only a general rule of thumb of sampling as densely as possible. The lack of understanding of what actually constitutes good views for NeRF makes it difficult to actively plan a sequence of views that yield the maximal reconstruction quality. We propose Surrogate Objectives for Active Radiance Fields (SOAR), which is a set of interpretable functions that evaluates the goodness of views using geometric and photometric visual cues - surface coverage, geometric complexity, textural complexity, and ray diversity. Moreover, by learning to infer the SOAR scores from a deep network, SOARNet, we are able to effectively select views in mere seconds instead of hours, without the need for prior visits to all the candidate views or training any radiance field during such planning. Our experiments show SOARNet outperforms the baselines with ~80x speed-up while achieving better or comparable reconstruction qualities. We finally show that SOAR is model-agnostic, thus it generalizes across fully neural-implicit to fully explicit approaches.