# Key Point-based Orientation Estimation of Strawberries for Robotic Fruit Picking

arXiv: https://arxiv.org/abs/2310.11333 · HTML: https://mlsanity.com/paper/2310.11333/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 27 references that are in the map.
- First posted on arXiv 17 Oct 2023.

## Builds on (most cited first, 8 of 8)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017)
- [Self-supervised 6D Object Pose Estimation for Robot Manipulation](https://mlsanity.com/paper/1909.10159/) (2019)
- [GPV-Pose: Category-level Object Pose Estimation via Geometry-guided Point-wise Voting](https://mlsanity.com/paper/2203.07918/) (2022)
- [Coupled Iterative Refinement for 6D Multi-Object Pose Estimation](https://mlsanity.com/paper/2204.12516/) (2022)
- [PoET: Pose Estimation Transformer for Single-View, Multi-Object 6D Pose Estimation](https://mlsanity.com/paper/2211.14125/) (2022)
- [Sim2Real Object-Centric Keypoint Detection and Description](https://mlsanity.com/paper/2202.00448/) (2022)

## Abstract (arXiv)

Selective robotic harvesting is a promising technological solution to address labour shortages which are affecting modern agriculture in many parts of the world. For an accurate and efficient picking process, a robotic harvester requires the precise location and orientation of the fruit to effectively plan the trajectory of the end effector. The current methods for estimating fruit orientation employ either complete 3D information which typically requires registration from multiple views or rely on fully-supervised learning techniques, which require difficult-to-obtain manual annotation of the reference orientation. In this paper, we introduce a novel key-point-based fruit orientation estimation method allowing for the prediction of 3D orientation from 2D images directly. The proposed technique can work without full 3D orientation annotations but can also exploit such information for improved accuracy. We evaluate our work on two separate datasets of strawberry images obtained from real-world data collection scenarios. Our proposed method achieves state-of-the-art performance with an average error as low as 8circ, improving predictions by ~30% compared to previous work presented in~citewagner2021efficient. Furthermore, our method is suited for real-time robotic applications with fast inference times of ~30ms.