# Semantic-aware Next-Best-View for Multi-DoFs Mobile System in Search-and-Acquisition based Visual Perception

arXiv: https://arxiv.org/abs/2404.16507 · HTML: https://mlsanity.com/paper/2404.16507/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 40 references that are in the map.
- First posted on arXiv 25 Apr 2024.

## Builds on (most cited first, 9 of 9)

- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Items](https://mlsanity.com/paper/2204.11918/) (2022)
- [Volumetric Instance-Aware Semantic Mapping and 3D Object Discovery](https://mlsanity.com/paper/1903.00268/) (2019)
- [NeU-NBV: Next Best View Planning Using Uncertainty Estimation in Image-Based Neural Rendering](https://mlsanity.com/paper/2303.01284/) (2023)
- [An Efficient Sampling-based Method for Online Informative Path Planning in Unknown Environments](https://mlsanity.com/paper/1909.09548/) (2019)
- [Autonomous UAV Exploration of Dynamic Environments via Incremental Sampling and Probabilistic Roadmap](https://mlsanity.com/paper/2010.07429/) (2020)
- [Online Next-Best-View Planner for 3D-Exploration and Inspection With a Mobile Manipulator Robot](https://mlsanity.com/paper/2203.10113/) (2022)
- [A Multi-Resolution Frontier-Based Planner for Autonomous 3D Exploration](https://mlsanity.com/paper/2011.02182/) (2020)
- [NBV-SC: Next Best View Planning based on Shape Completion for Fruit Mapping and Reconstruction](https://mlsanity.com/paper/2209.15376/) (2022)

## Built on by (most cited first, 1 of 1)

- [Boundary Exploration of Next Best View Policy in 3D Robotic Scanning](https://mlsanity.com/paper/2412.10444/) (2024)

## Abstract (arXiv)

Efficient visual perception using mobile systems is crucial, particularly in unknown environments such as search and rescue operations, where swift and comprehensive perception of objects of interest is essential. In such real-world applications, objects of interest are often situated in complex environments, making the selection of the 'Next Best' view based solely on maximizing visibility gain suboptimal. Semantics, providing a higher-level interpretation of perception, should significantly contribute to the selection of the next viewpoint for various perception tasks. In this study, we formulate a novel information gain that integrates both visibility gain and semantic gain in a unified form to select the semantic-aware Next-Best-View. Additionally, we design an adaptive strategy with termination criterion to support a two-stage search-and-acquisition manoeuvre on multiple objects of interest aided by a multi-degree-of-freedoms (Multi-DoFs) mobile system. Several semantically relevant reconstruction metrics, including perspective directivity and region of interest (ROI)-to-full reconstruction volume ratio, are introduced to evaluate the performance of the proposed approach. Simulation experiments demonstrate the advantages of the proposed approach over existing methods, achieving improvements of up to 27.13% for the ROI-to-full reconstruction volume ratio and a 0.88234 average perspective directivity. Furthermore, the planned motion trajectory exhibits better perceiving coverage toward the target.