# Viewpoint Planning for Fruit Size and Position Estimation

arXiv: https://arxiv.org/abs/2011.00275 · HTML: https://mlsanity.com/paper/2011.00275/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2020.

## Built on by (most cited first, 9 of 9)

- [A Review on Viewpoints and Path-planning for UAV-based 3D Reconstruction](https://mlsanity.com/paper/2205.03716/) (2022)
- [Attention-driven Next-best-view Planning for Efficient Reconstruction of Plants and Targeted Plant Parts](https://mlsanity.com/paper/2206.10274/) (2022)
- [NBV-SC: Next Best View Planning based on Shape Completion for Fruit Mapping and Reconstruction](https://mlsanity.com/paper/2209.15376/) (2022)
- [Immature Green Apple Detection and Sizing in Commercial Orchards using YOLOv8 and Shape Fitting Techniques](https://mlsanity.com/paper/2401.08629/) (2023)
- [Semantics-Aware Next-best-view Planning for Efficient Search and Detection of Task-relevant Plant Parts](https://mlsanity.com/paper/2306.09801/) (2023)
- [Gradient-based Local Next-best-view Planning for Improved Perception of Targeted Plant Nodes](https://mlsanity.com/paper/2311.16759/) (2023)
- [Autonomous Apple Fruitlet Sizing and Growth Rate Tracking using Computer Vision](https://mlsanity.com/paper/2212.01506/) (2022)
- [STAIR: Semantic-Targeted Active Implicit Reconstruction](https://mlsanity.com/paper/2403.11233/) (2024)
- [SSL-NBV: A Self-Supervised-Learning-Based Next-Best-View algorithm for Efficient 3D Plant Reconstruction by a Robot](https://mlsanity.com/paper/2410.14790/) (2024)

## Abstract (arXiv)

Modern agricultural applications require knowledge about the position and size of fruits on plants. However, occlusions from leaves typically make obtaining this information difficult. We present a novel viewpoint planning approach that builds up an octree of plants with labeled regions of interest (ROIs), i.e., fruits. Our method uses this octree to sample viewpoint candidates that increase the information around the fruit regions and evaluates them using a heuristic utility function that takes into account the expected information gain. Our system automatically switches between ROI targeted sampling and exploration sampling, which considers general frontier voxels, depending on the estimated utility. When the plants have been sufficiently covered with the RGB-D sensor, our system clusters the ROI voxels and estimates the position and size of the detected fruits. We evaluated our approach in simulated scenarios and compared the resulting fruit estimations with the ground truth. The results demonstrate that our combined approach outperforms a sampling method that does not explicitly consider the ROIs to generate viewpoints in terms of the number of discovered ROI cells. Furthermore, we show the real-world applicability by testing our framework on a robotic arm equipped with an RGB-D camera installed on an automated pipe-rail trolley in a capsicum glasshouse.