# Gradient-based Local Next-best-view Planning for Improved Perception of Targeted Plant Nodes

arXiv: https://arxiv.org/abs/2311.16759 · HTML: https://mlsanity.com/paper/2311.16759/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 27 references that are in the map.
- First posted on arXiv 28 Nov 2023.

## Builds on (most cited first, 9 of 9)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [A Survey on Active Simultaneous Localization and Mapping: State of the Art and New Frontiers](https://mlsanity.com/paper/2207.00254/) (2022)
- [Fast Frontier-based Information-driven Autonomous Exploration with an MAV](https://mlsanity.com/paper/2002.04440/) (2020)
- [Adaptive Informative Path Planning Using Deep Reinforcement Learning for UAV-based Active Sensing](https://mlsanity.com/paper/2109.13570/) (2021)
- [Viewpoint Planning for Fruit Size and Position Estimation](https://mlsanity.com/paper/2011.00275/) (2020)
- [Fruit Mapping with Shape Completion for Autonomous Crop Monitoring](https://mlsanity.com/paper/2203.15489/) (2022)
- [Frontier-based Automatic-differentiable Information Gain Measure for Robotic Exploration of Unknown 3D Environments](https://mlsanity.com/paper/2011.05288/) (2020)

## Built on by (most cited first, 3 of 3)

- [Observe Then Act: Asynchronous Active Vision-Action Model for Robotic Manipulation](https://mlsanity.com/paper/2409.14891/) (2024)
- [DNRSelect: Active Best View Selection for Deferred Neural Rendering](https://mlsanity.com/paper/2501.12150/) (2025)
- [DAV-GSWT: Diffusion-Active-View Sampling for Data-Efficient Gaussian Splatting Wang Tiles](https://mlsanity.com/paper/2602.15355/) (2026)

## Abstract (arXiv)

Robots are increasingly used in tomato greenhouses to automate labour-intensive tasks such as selective harvesting and de-leafing. To perform these tasks, robots must be able to accurately and efficiently perceive the plant nodes that need to be cut, despite the high levels of occlusion from other plant parts. We formulate this problem as a local next-best-view (NBV) planning task where the robot has to plan an efficient set of camera viewpoints to overcome occlusion and improve the quality of perception. Our formulation focuses on quickly improving the perception accuracy of a single target node to maximise its chances of being cut. Previous methods of NBV planning mostly focused on global view planning and used random sampling of candidate viewpoints for exploration, which could suffer from high computational costs, ineffective view selection due to poor candidates, or non-smooth trajectories due to inefficient sampling. We propose a gradient-based NBV planner using differential ray sampling, which directly estimates the local gradient direction for viewpoint planning to overcome occlusion and improve perception. Through simulation experiments, we showed that our planner can handle occlusions and improve the 3D reconstruction and position estimation of nodes equally well as a sampling-based NBV planner, while taking ten times less computation and generating 28% more efficient trajectories.