# 3D Pose Estimation of Tomato Peduncle Nodes using Deep Keypoint Detection and Point Cloud

arXiv: https://arxiv.org/abs/2311.04699 · HTML: https://mlsanity.com/paper/2311.04699/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 48 references that are in the map.
- First posted on arXiv 8 Nov 2023.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Deep learning in agriculture: A survey](https://mlsanity.com/paper/1807.11809/) (2018)
- [Deep Fruit Detection in Orchards](https://mlsanity.com/paper/1610.03677/) (2016)
- [Supervised Learning of the Next-Best-View for 3D Object Reconstruction](https://mlsanity.com/paper/1905.05833/) (2019)
- [Grape detection, segmentation and tracking using deep neural networks and three-dimensional association](https://mlsanity.com/paper/1907.11819/) (2019)
- [T-LEAP: Occlusion-robust pose estimation of walking cows using temporal information](https://mlsanity.com/paper/2104.08029/) (2021)
- [Fruit Detection, Segmentation and 3D Visualisation of Environments in Apple Orchards](https://mlsanity.com/paper/1911.12889/) (2019)
- [Attention-driven Next-best-view Planning for Efficient Reconstruction of Plants and Targeted Plant Parts](https://mlsanity.com/paper/2206.10274/) (2022)
- [Peduncle Detection of Sweet Pepper for Autonomous Crop Harvesting - Combined Colour and 3D Information](https://mlsanity.com/paper/1701.08608/) (2017)

## Abstract (arXiv)

Greenhouse production of fruits and vegetables in developed countries is challenged by labor 12 scarcity and high labor costs. Robots offer a good solution for sustainable and cost-effective 13 production. Acquiring accurate spatial information about relevant plant parts is vital for 14 successful robot operation. Robot perception in greenhouses is challenging due to variations in 15 plant appearance, viewpoints, and illumination. This paper proposes a keypoint-detection-based 16 method using data from an RGB-D camera to estimate the 3D pose of peduncle nodes, which 17 provides essential information to harvest the tomato bunches. 18 19 Specifically, this paper proposes a method that detects four anatomical landmarks in the color 20 image and then integrates 3D point-cloud information to determine the 3D pose. A 21 comprehensive evaluation was conducted in a commercial greenhouse to gain insight into the 22 performance of different parts of the method. The results showed: (1) high accuracy in object 23 detection, achieving an Average Precision (AP) of AP@0.5=0.96; (2) an average Percentage of 24 Detected Joints (PDJ) of the keypoints of PhDJ@0.2=94.31%; and (3) 3D pose estimation 25 accuracy with mean absolute errors (MAE) of 11.38o and 9.93o for the relative upper and lower 26 angles between the peduncle and main stem, respectively. Furthermore, the capability to handle 27 variations in viewpoint was investigated, demonstrating the method was robust to view changes. 28 However, canonical and higher views resulted in slightly higher performance compared to other 29 views. Although tomato was selected as a use case, the proposed method is also applicable to 30 other greenhouse crops like pepper.