# Next-Best-View Estimation based on Deep Reinforcement Learning for Active Object Classification

arXiv: https://arxiv.org/abs/2110.06766 · HTML: https://mlsanity.com/paper/2110.06766/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 37 references that are in the map.
- First posted on arXiv 13 Oct 2021.

## Builds on (most cited first, 13 of 13)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Reinforcement Learning with Deep Energy-Based Policies](https://mlsanity.com/paper/1702.08165/) (2017)
- [Multi-view Convolutional Neural Networks for 3D Shape Recognition](https://mlsanity.com/paper/1505.00880/) (2015)
- [Revisiting Active Perception](https://mlsanity.com/paper/1603.02729/) (2016)
- [Recovering 6D Object Pose and Predicting Next-Best-View in the Crowd](https://mlsanity.com/paper/1512.07506/) (2015)
- [Benchmarking Reinforcement Learning Algorithms on Real-World Robots](https://mlsanity.com/paper/1809.07731/) (2018)
- [Next-Best View Policy for 3D Reconstruction](https://mlsanity.com/paper/2008.12664/) (2020)
- [3D ShapeNets: A Deep Representation for Volumetric Shapes](https://mlsanity.com/paper/1406.5670/) (2014)
- [Supervised Learning of the Next-Best-View for 3D Object Reconstruction](https://mlsanity.com/paper/1905.05833/) (2019)
- [Blocks World Revisited: The Effect of Self-Occlusion on Classification by Convolutional Neural Networks](https://mlsanity.com/paper/2102.12911/) (2021)
- [Learning Local RGB-to-CAD Correspondences for Object Pose Estimation](https://mlsanity.com/paper/1811.07249/) (2018)

## Abstract (arXiv)

The presentation and analysis of image data from a single viewpoint are often not sufficient to solve a task. Several viewpoints are necessary to obtain more information. The next-best-view problem attempts to find the optimal viewpoint with the greatest information gain for the underlying task. In this work, a robot arm holds an object in its end-effector and searches for a sequence of next-best-view to explicitly identify the object. We use Soft Actor-Critic (SAC), a method of deep reinforcement learning, to learn these next-best-views for a specific set of objects. The evaluation shows that an agent can learn to determine an object pose to which the robot arm should move an object. This leads to a viewpoint that provides a more accurate prediction to distinguish such an object from other objects better. We make the code publicly available for the scientific community and for reproducibility.