# Active Perception using Neural Radiance Fields

arXiv: https://arxiv.org/abs/2310.09892 · HTML: https://mlsanity.com/paper/2310.09892/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Oct 2023.

## Built on by (most cited first, 8 of 8)

- [Gaussian Splatting to Real World Flight Navigation Transfer with Liquid Networks](https://mlsanity.com/paper/2406.15149/) (2024)
- [Understanding while Exploring: Semantics-driven Active Mapping](https://mlsanity.com/paper/2506.00225/) (2025)
- [STAIR: Semantic-Targeted Active Implicit Reconstruction](https://mlsanity.com/paper/2403.11233/) (2024)
- [Real-World Reinforcement Learning of Active Perception Behaviors](https://mlsanity.com/paper/2512.01188/) (2025)
- [Active Scout: Multi-Target Tracking Using Neural Radiance Fields in Dense Urban Environments](https://mlsanity.com/paper/2406.07431/) (2024)
- [TRACE: Ergodic Trajectory Optimization for Active Scene Reconstruction](https://mlsanity.com/paper/2608.02304/) (2026)
- [Multi-Scale Semantic Mapping in Urban Environments via Observation Calibration and Policy Dependence Regularization](https://mlsanity.com/paper/2609.34833/) (2026)
- [VCURF: Virtual Camera-based Uncertainty of Radiance Fields](https://mlsanity.com/paper/2610.04076/) (2026)

## Abstract (arXiv)

We study active perception from first principles to argue that an autonomous agent performing active perception should maximize the mutual information that past observations posses about future ones. Doing so requires (a) a representation of the scene that summarizes past observations and the ability to update this representation to incorporate new observations (state estimation and mapping), (b) the ability to synthesize new observations of the scene (a generative model), and (c) the ability to select control trajectories that maximize predictive information (planning). This motivates a neural radiance field (NeRF)-like representation which captures photometric, geometric and semantic properties of the scene grounded. This representation is well-suited to synthesizing new observations from different viewpoints. And thereby, a sampling-based planner can be used to calculate the predictive information from synthetic observations along dynamically-feasible trajectories. We use active perception for exploring cluttered indoor environments and employ a notion of semantic uncertainty to check for the successful completion of an exploration task. We demonstrate these ideas via simulation in realistic 3D indoor environments.