# DNRSelect: Active Best View Selection for Deferred Neural Rendering

arXiv: https://arxiv.org/abs/2501.12150 · HTML: https://mlsanity.com/paper/2501.12150/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in RL with diffusion and flow models, one of 15 directions in Deep reinforcement learning methods.
- Builds on 33 of its 46 references that are in the map.
- First posted on arXiv 21 Jan 2025.

## Placed in

- Deep reinforcement learning methods / [RL with diffusion and flow models](https://mlsanity.com/deeprl/rl-with-diffusion-and-flow-models/)

## Builds on (most cited first, 30 of 33)

- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [TensoRF: Tensorial Radiance Fields](https://mlsanity.com/paper/2203.09517/) (2022)
- [Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering](https://mlsanity.com/paper/2312.00109/) (2023)
- [Advances in Neural Rendering](https://mlsanity.com/paper/2111.05849/) (2021)
- [State of the Art on Neural Rendering](https://mlsanity.com/paper/2004.03805/) (2020)
- [Deferred Neural Rendering: Image Synthesis using Neural Textures](https://mlsanity.com/paper/1904.12356/) (2019)
- [GaussianPro: 3D Gaussian Splatting with Progressive Propagation](https://mlsanity.com/paper/2402.14650/) (2024)
- [ActiveNeRF: Learning where to See with Uncertainty Estimation](https://mlsanity.com/paper/2209.08546/) (2022)
- [textitS3Gaussian: Self-Supervised Street Gaussians for Autonomous Driving](https://mlsanity.com/paper/2405.20323/) (2024)
- [Bayes' Rays: Uncertainty Quantification for Neural Radiance Fields](https://mlsanity.com/paper/2309.03185/) (2023)
- [CG-SLAM: Efficient Dense RGB-D SLAM in a Consistent Uncertainty-aware 3D Gaussian Field](https://mlsanity.com/paper/2403.16095/) (2024)
- [Density-aware NeRF Ensembles: Quantifying Predictive Uncertainty in Neural Radiance Fields](https://mlsanity.com/paper/2209.08718/) (2022)
- [ANR: Articulated Neural Rendering for Virtual Avatars](https://mlsanity.com/paper/2012.12890/) (2020)
- [NeU-NBV: Next Best View Planning Using Uncertainty Estimation in Image-Based Neural Rendering](https://mlsanity.com/paper/2303.01284/) (2023)
- [Multi-View Mesh Reconstruction with Neural Deferred Shading](https://mlsanity.com/paper/2212.04386/) (2022)
- [DeferredGS: Decoupled and Editable Gaussian Splatting with Deferred Shading](https://mlsanity.com/paper/2404.09412/) (2024)
- [I-MedSAM: Implicit Medical Image Segmentation with Segment Anything](https://mlsanity.com/paper/2311.17081/) (2023)
- [PLGS: Robust Panoptic Lifting with 3D Gaussian Splatting](https://mlsanity.com/paper/2410.17505/) (2024)
- [Multi-View Picking: Next-best-view Reaching for Improved Grasping in Clutter](https://mlsanity.com/paper/1809.08564/) (2018)
- [NeRF Director: Revisiting View Selection in Neural Volume Rendering](https://mlsanity.com/paper/2406.08839/) (2024)
- [Online Next-Best-View Planner for 3D-Exploration and Inspection With a Mobile Manipulator Robot](https://mlsanity.com/paper/2203.10113/) (2022)
- [NTO3D: Neural Target Object 3D Reconstruction with Segment Anything](https://mlsanity.com/paper/2309.12790/) (2023)
- [EMD: Explicit Motion Modeling for High-Quality Street Gaussian Splatting](https://mlsanity.com/paper/2411.15582/) (2024)
- [GraphAvatar: Compact Head Avatars with GNN-Generated 3D Gaussians](https://mlsanity.com/paper/2412.13983/) (2024)
- [NeRF-NQA: No-Reference Quality Assessment for Scenes Generated by NeRF and Neural View Synthesis Methods](https://mlsanity.com/paper/2412.08029/) (2024)
- [GazeGaussian: High-Fidelity Gaze Redirection with 3D Gaussian Splatting](https://mlsanity.com/paper/2411.12981/) (2024)

## Built on by (most cited first, 2 of 2)

- [Next Best View Selections for Semantic and Dynamic 3D Gaussian Splatting](https://mlsanity.com/paper/2512.22771/) (2025)
- [GraspFoM: Towards Reconstruction-Driven Robotic Grasping with 3D Foundation Priors](https://mlsanity.com/paper/2606.08440/) (2026)

## Abstract (arXiv)

Deferred neural rendering (DNR) is an emerging computer graphics pipeline designed for high-fidelity rendering and robotic perception. However, DNR heavily relies on datasets composed of numerous ray-traced images and demands substantial computational resources. It remains under-explored how to reduce the reliance on high-quality ray-traced images while maintaining the rendering fidelity. In this paper, we propose DNRSelect, which integrates a reinforcement learning-based view selector and a 3D texture aggregator for deferred neural rendering. We first propose a novel view selector for deferred neural rendering based on reinforcement learning, which is trained on easily obtained rasterized images to identify the optimal views. By acquiring only a few ray-traced images for these selected views, the selector enables DNR to achieve high-quality rendering. To further enhance spatial awareness and geometric consistency in DNR, we introduce a 3D texture aggregator that fuses pyramid features from depth maps and normal maps with UV maps. Given that acquiring ray-traced images is more time-consuming than generating rasterized images, DNRSelect minimizes the need for ray-traced data by using only a few selected views while still achieving high-fidelity rendering results. We conduct detailed experiments and ablation studies on the NeRF-Synthetic dataset to demonstrate the effectiveness of DNRSelect. The code will be released.