# Next-best-view Regression using a 3D Convolutional Neural Network

arXiv: https://arxiv.org/abs/2101.09397 · HTML: https://mlsanity.com/paper/2101.09397/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 43 references that are in the map.
- First posted on arXiv 23 Jan 2021.

## Builds on (most cited first, 9 of 9)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [PCN: Point Completion Network](https://mlsanity.com/paper/1808.00671/) (2018)
- [Recovering 6D Object Pose and Predicting Next-Best-View in the Crowd](https://mlsanity.com/paper/1512.07506/) (2015)
- [Supervised Learning of the Next-Best-View for 3D Object Reconstruction](https://mlsanity.com/paper/1905.05833/) (2019)
- [Pairwise Decomposition of Image Sequences for Active Multi-View Recognition](https://mlsanity.com/paper/1605.08359/) (2016)
- [Learn-to-Score: Efficient 3D Scene Exploration by Predicting View Utility](https://mlsanity.com/paper/1806.10354/) (2018)
- [Multi-Sensor Next-Best-View Planning as Matroid-Constrained Submodular Maximization](https://mlsanity.com/paper/2007.02084/) (2020)
- [A Motion Planning Strategy for the Active Vision-Based Mapping of Ground-Level Structures](https://mlsanity.com/paper/1602.06667/) (2016)

## Built on by (most cited first, 4 of 4)

- [MACARONS: Mapping And Coverage Anticipation with RGB Online Self-Supervision](https://mlsanity.com/paper/2303.03315/) (2023)
- [SCONE: Surface Coverage Optimization in Unknown Environments by Volumetric Integration](https://mlsanity.com/paper/2208.10449/) (2022)
- [Uncertainty-Driven Active Vision for Implicit Scene Reconstruction](https://mlsanity.com/paper/2210.00978/) (2022)
- [Next Best View For Point-Cloud Model Acquisition: Bayesian Approximation and Uncertainty Analysis](https://mlsanity.com/paper/2411.01734/) (2024)

## Abstract (arXiv)

Automated three-dimensional (3D) object reconstruction is the task of building a geometric representation of a physical object by means of sensing its surface. Even though new single view reconstruction techniques can predict the surface, they lead to incomplete models, specially, for non commons objects such as antique objects or art sculptures. Therefore, to achieve the task's goals, it is essential to automatically determine the locations where the sensor will be placed so that the surface will be completely observed. This problem is known as the next-best-view problem. In this paper, we propose a data-driven approach to address the problem. The proposed approach trains a 3D convolutional neural network (3D CNN) with previous reconstructions in order to regress the btxtposition of the next-best-view. To the best of our knowledge, this is one of the first works that directly infers the next-best-view in a continuous space using a data-driven approach for the 3D object reconstruction task. We have validated the proposed approach making use of two groups of experiments. In the first group, several variants of the proposed architecture are analyzed. Predicted next-best-views were observed to be closely positioned to the ground truth. In the second group of experiments, the proposed approach is requested to reconstruct several unseen objects, namely, objects not considered by the 3D CNN during training nor validation. Coverage percentages of up to 90 % were observed. With respect to current state-of-the-art methods, the proposed approach improves the performance of previous next-best-view classification approaches and it is quite fast in running time (3 frames per second), given that it does not compute the expensive ray tracing required by previous information metrics.