# Active Scout: Multi-Target Tracking Using Neural Radiance Fields in Dense Urban Environments

arXiv: https://arxiv.org/abs/2406.07431 · HTML: https://mlsanity.com/paper/2406.07431/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 32 references that are in the map.
- First posted on arXiv 11 Jun 2024.

## Builds on (most cited first, 17 of 17)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Block-NeRF: Scalable Large Scene Neural View Synthesis](https://mlsanity.com/paper/2202.05263/) (2022)
- [Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation](https://mlsanity.com/paper/2312.02145/) (2023)
- [BungeeNeRF: Progressive Neural Radiance Field for Extreme Multi-scale Scene Rendering](https://mlsanity.com/paper/2112.05504/) (2021)
- [Revisiting Active Perception](https://mlsanity.com/paper/1603.02729/) (2016)
- [NerfAcc: A General NeRF Acceleration Toolbox](https://mlsanity.com/paper/2210.04847/) (2022)
- [FisherRF: Active View Selection and Uncertainty Quantification for Radiance Fields using Fisher Information](https://mlsanity.com/paper/2311.17874/) (2023)
- [Density-aware NeRF Ensembles: Quantifying Predictive Uncertainty in Neural Radiance Fields](https://mlsanity.com/paper/2209.08718/) (2022)
- [Efficient Informative Sensing using Multiple Robots](https://mlsanity.com/paper/1401.3462/) (2014)
- [Scalable Reinforcement Learning Policies for Multi-Agent Control](https://mlsanity.com/paper/2011.08055/) (2020)
- [Graph Neural Networks for Multi-Robot Active Information Acquisition](https://mlsanity.com/paper/2209.12091/) (2022)
- [NerfBridge: Bringing Real-time, Online Neural Radiance Field Training to Robotics](https://mlsanity.com/paper/2305.09761/) (2023)
- [RotorPy: A Python-based Multirotor Simulator with Aerodynamics for Education and Research](https://mlsanity.com/paper/2306.04485/) (2023)
- [Active Perception using Neural Radiance Fields](https://mlsanity.com/paper/2310.09892/) (2023)
- [Decentralized Multi-Agent Active Search and Tracking when Targets Outnumber Agents](https://mlsanity.com/paper/2401.03154/) (2024)
- [Obstructed Target Tracking in Urban Environments](https://mlsanity.com/paper/1906.02397/) (2019)

## Abstract (arXiv)

We study pursuit-evasion games in highly occluded urban environments, e.g. tall buildings in a city, where a scout (quadrotor) tracks multiple dynamic targets on the ground. We show that we can build a neural radiance field (NeRF) representation of the city -- online -- using RGB and depth images from different vantage points. This representation is used to calculate the information gain to both explore unknown parts of the city and track the targets -- thereby giving a completely first-principles approach to actively tracking dynamic targets. We demonstrate, using a custom-built simulator using Open Street Maps data of Philadelphia and New York City, that we can explore and locate 20 stationary targets within 300 steps. This is slower than a greedy baseline, which does not use active perception. But for dynamic targets that actively hide behind occlusions, we show that our approach maintains, at worst, a tracking error of 200m; the greedy baseline can have a tracking error as large as 600m. We observe a number of interesting properties in the scout's policies, e.g., it switches its attention to track a different target periodically, as the quality of the NeRF representation improves over time, the scout also becomes better in terms of target tracking. Code is available at https://github.com/grasp-lyrl/ActiveScout.