# 3D shape sensing and deep learning-based segmentation of strawberries

arXiv: https://arxiv.org/abs/2111.13663 · HTML: https://mlsanity.com/paper/2111.13663/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 54 references that are in the map.
- First posted on arXiv 26 Nov 2021.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [Joint 2D-3D-Semantic Data for Indoor Scene Understanding](https://mlsanity.com/paper/1702.01105/) (2017)
- [PointSIFT: A SIFT-like Network Module for 3D Point Cloud Semantic Segmentation](https://mlsanity.com/paper/1807.00652/) (2018)
- [3D Fully Convolutional Network for Vehicle Detection in Point Cloud](https://mlsanity.com/paper/1611.08069/) (2016)
- [Agricultural Robotics: The Future of Robotic Agriculture](https://mlsanity.com/paper/1806.06762/) (2018)
- [Training and Testing Object Detectors with Virtual Images](https://mlsanity.com/paper/1712.08470/) (2017)
- [A Sweet Pepper Harvesting Robot for Protected Cropping Environments](https://mlsanity.com/paper/1810.11920/) (2018)

## Built on by (most cited first, 2 of 2)

- [PLANesT-3D: A new annotated dataset for segmentation of 3D plant point clouds](https://mlsanity.com/paper/2407.21150/) (2024)
- [Phenology-based learning framework for yield estimation and harvest forecasting of raspberry fruits](https://mlsanity.com/paper/2411.00967/) (2024)

## Abstract (arXiv)

Automation and robotisation of the agricultural sector are seen as a viable solution to socio-economic challenges faced by this industry. This technology often relies on intelligent perception systems providing information about crops, plants and the entire environment. The challenges faced by traditional 2D vision systems can be addressed by modern 3D vision systems which enable straightforward localisation of objects, size and shape estimation, or handling of occlusions. So far, the use of 3D sensing was mainly limited to indoor or structured environments. In this paper, we evaluate modern sensing technologies including stereo and time-of-flight cameras for 3D perception of shape in agriculture and study their usability for segmenting out soft fruit from background based on their shape. To that end, we propose a novel 3D deep neural network which exploits the organised nature of information originating from the camera-based 3D sensors. We demonstrate the superior performance and efficiency of the proposed architecture compared to the state-of-the-art 3D networks. Through a simulated study, we also show the potential of the 3D sensing paradigm for object segmentation in agriculture and provide insights and analysis of what shape quality is needed and expected for further analysis of crops. The results of this work should encourage researchers and companies to develop more accurate and robust 3D sensing technologies to assure their wider adoption in practical agricultural applications.