# Next-Best-View Prediction for Active Stereo Cameras and Highly Reflective Objects

arXiv: https://arxiv.org/abs/2202.13263 · HTML: https://mlsanity.com/paper/2202.13263/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 43 references that are in the map.
- First posted on arXiv 27 Feb 2022.

## Builds on (most cited first, 14 of 14)

- [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017)
- [DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion](https://mlsanity.com/paper/1901.04780/) (2019)
- [BOP: Benchmark for 6D Object Pose Estimation](https://mlsanity.com/paper/1808.08319/) (2018)
- [Implicit 3D Orientation Learning for 6D Object Detection from RGB Images](https://mlsanity.com/paper/1902.01275/) (2019)
- [ClearGrasp: 3D Shape Estimation of Transparent Objects for Manipulation](https://mlsanity.com/paper/1910.02550/) (2019)
- [Deep Depth Completion of a Single RGB-D Image](https://mlsanity.com/paper/1803.09326/) (2018)
- [Revisiting Active Perception](https://mlsanity.com/paper/1603.02729/) (2016)
- [ROBI: A Multi-View Dataset for Reflective Objects in Robotic Bin-Picking](https://mlsanity.com/paper/2105.04112/) (2021)
- [GapFlyt: Active Vision Based Minimalist Structure-less Gap Detection For Quadrotor Flight](https://mlsanity.com/paper/1802.05330/) (2018)
- [Active 6D Multi-Object Pose Estimation in Cluttered Scenarios with Deep Reinforcement Learning](https://mlsanity.com/paper/1910.08811/) (2019)
- [Object SLAM-Based Active Mapping and Robotic Grasping](https://mlsanity.com/paper/2012.01788/) (2020)
- [Probabilistic Multi-View Fusion of Active Stereo Depth Maps for Robotic Bin-Picking](https://mlsanity.com/paper/2103.10968/) (2021)
- [Pose Estimation for Texture-less Shiny Objects in a Single RGB Image Using Synthetic Training Data](https://mlsanity.com/paper/1909.10270/) (2019)
- [Self-Guided Instance-Aware Network for Depth Completion and Enhancement](https://mlsanity.com/paper/2105.12186/) (2021)

## Built on by (most cited first, 4 of 4)

- [A Low-Cost, High-Speed, and Robust Bin Picking System for Factory Automation Enabled by a Non-Stop, Multi-View, and Active Vision Scheme](https://mlsanity.com/paper/2410.00706/) (2024)
- [Active 6D Pose Estimation for Textureless Objects using Multi-View RGB Frames](https://mlsanity.com/paper/2503.03726/) (2025)
- [Reasoning and Learning a Perceptual Metric for Self-Training of Reflective Objects in Bin-Picking with a Low-cost Camera](https://mlsanity.com/paper/2503.20207/) (2025)
- [DKPMV: Dense Keypoints Fusion from Multi-View RGB Frames for 6D Pose Estimation of Textureless Objects](https://mlsanity.com/paper/2510.10933/) (2025)

## Abstract (arXiv)

Depth acquisition with the active stereo camera is a challenging task for highly reflective objects. When setup permits, multi-view fusion can provide increased levels of depth completion. However, due to the slow acquisition speed of high-end active stereo cameras, collecting a large number of viewpoints for a single scene is generally not practical. In this work, we propose a next-best-view framework to strategically select camera viewpoints for completing depth data on reflective objects. In particular, we explicitly model the specular reflection of reflective surfaces based on the Phong reflection model and a photometric response function. Given the object CAD model and grayscale image, we employ an RGB-based pose estimator to obtain current pose predictions from the existing data, which is used to form predicted surface normal and depth hypotheses, and allows us to then assess the information gain from a subsequent frame for any candidate viewpoint. Using this formulation, we implement an active perception pipeline which is evaluated on a challenging real-world dataset. The evaluation results demonstrate that our active depth acquisition method outperforms two strong baselines for both depth completion and object pose estimation performance.