# Self-Supervised Learning for Robotic Leaf Manipulation: A Hybrid Geometric-Neural Approach

arXiv: https://arxiv.org/abs/2505.03702 · HTML: https://mlsanity.com/paper/2505.03702/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 59 references that are in the map.
- First posted on arXiv 6 May 2025.

## Builds on (most cited first, 11 of 11)

- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching](https://mlsanity.com/paper/2109.07547/) (2021)
- [Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics](https://mlsanity.com/paper/1703.09312/) (2017)
- [Robotic Pick-and-Place of Novel Objects in Clutter with Multi-Affordance Grasping and Cross-Domain Image Matching](https://mlsanity.com/paper/1710.01330/) (2017)
- [DoubleU-Net: A Deep Convolutional Neural Network for Medical Image Segmentation](https://mlsanity.com/paper/2006.04868/) (2020)
- [Closing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approach](https://mlsanity.com/paper/1804.05172/) (2018)
- [Self-Supervised Model Adaptation for Multimodal Semantic Segmentation](https://mlsanity.com/paper/1808.03833/) (2018)
- [Agricultural Robotics: The Future of Robotic Agriculture](https://mlsanity.com/paper/1806.06762/) (2018)
- [Autonomous Sweet Pepper Harvesting for Protected Cropping Systems](https://mlsanity.com/paper/1706.02023/) (2017)
- [Peduncle Detection of Sweet Pepper for Autonomous Crop Harvesting - Combined Colour and 3D Information](https://mlsanity.com/paper/1701.08608/) (2017)
- [An In-field Automatic Wheat Disease Diagnosis System](https://mlsanity.com/paper/1710.08299/) (2017)

## Built on by (most cited first, 2 of 2)

- [Find the Fruit: Zero-Shot Sim2Real RL for Occlusion-Aware Plant Manipulation](https://mlsanity.com/paper/2505.16547/) (2025)
- [Deep Learning Perspective of Scene Understanding in Autonomous Robots](https://mlsanity.com/paper/2512.14020/) (2025)

## Abstract (arXiv)

Automating leaf manipulation in agricultural settings faces significant challenges, including the variability of plant morphologies and deformable leaves. We propose a novel hybrid geometric-neural approach for autonomous leaf grasping that combines traditional computer vision with neural networks through self-supervised learning. Our method integrates YOLOv8 for instance segmentation and RAFT-Stereo for 3D depth estimation to build rich leaf representations, which feed into both a geometric feature scoring pipeline and a neural refinement module (GraspPointCNN). The key innovation is our confidence-weighted fusion mechanism that dynamically balances the contribution of each approach based on prediction certainty. Our self-supervised framework uses the geometric pipeline as an expert teacher to automatically generate training data. Experiments demonstrate that our approach achieves an 88.0% success rate in controlled environments and 84.7% in real greenhouse conditions, significantly outperforming both purely geometric (75.3%) and neural (60.2%) methods. This work establishes a new paradigm for agricultural robotics where domain expertise is seamlessly integrated with machine learning capabilities, providing a foundation for fully automated crop monitoring systems.